2025-09-01 Mon 02:02:01:503.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: vhost interface name : vhost0 2025-09-01 Mon 02:02:01:503.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: vhost IP Address : 10.20.0.18/24 2025-09-01 Mon 02:02:01:503.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: vhost gateway : 0.0.0.0 2025-09-01 Mon 02:02:01:503.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Ethernet ports : ens6 2025-09-01 Mon 02:02:01:503.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Loopback IP : 0.0.0.0 2025-09-01 Mon 02:02:01:503.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Xmpp Servers : 10.20.0.25:5269 10.20.0.193:5269 10.20.0.129:5269 2025-09-01 Mon 02:02:01:503.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Xmpp Authentication : 0 2025-09-01 Mon 02:02:01:503.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Cluster-Name : 2025-09-01 Mon 02:02:01:503.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: DNS Servers : 10.20.0.25:53 10.20.0.193:53 10.20.0.129:53 2025-09-01 Mon 02:02:01:503.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: DNS client port : 8098 2025-09-01 Mon 02:02:01:503.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: DNS timeout : 3000 2025-09-01 Mon 02:02:01:503.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: DNS max retries : 2 2025-09-01 Mon 02:02:01:503.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Xmpp Dns Authentication : 0 2025-09-01 Mon 02:02:01:503.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TSN Servers : [] 2025-09-01 Mon 02:02:01:503.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: COLLECTOR Servers : 10.0.0.249:8086 10.0.0.38:8086 10.0.0.131:8086 2025-09-01 Mon 02:02:01:503.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Tunnel-Type : MPLSoGRE 2025-09-01 Mon 02:02:01:503.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Metadata-Proxy Shared Secret: contrail 2025-09-01 Mon 02:02:01:503.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Metadata-Proxy Port : 8097 2025-09-01 Mon 02:02:01:503.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Metadata-Proxy SSL Flag : 0 2025-09-01 Mon 02:02:01:503.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Linklocal Max System Flows : 16384 2025-09-01 Mon 02:02:01:503.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Linklocal Max Vm Flows : 16384 2025-09-01 Mon 02:02:01:503.746 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow cache timeout : 0 2025-09-01 Mon 02:02:01:503.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Stale Interface cleanup timeout : 60 2025-09-01 Mon 02:02:01:503.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow thread count : 1 2025-09-01 Mon 02:02:01:503.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow latency limit : 0 2025-09-01 Mon 02:02:01:503.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow index-mgr sm log count : 0 2025-09-01 Mon 02:02:01:503.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow add-tokens : 50 2025-09-01 Mon 02:02:01:503.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow ksync-tokens : 25 2025-09-01 Mon 02:02:01:503.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow del-tokens : 16 2025-09-01 Mon 02:02:01:503.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow update-tokens : 16 2025-09-01 Mon 02:02:01:503.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Pin flow netlink task to CPU: 2025-09-01 Mon 02:02:01:503.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Maximum sessions : 100 2025-09-01 Mon 02:02:01:503.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Maximum session aggregates : 8 2025-09-01 Mon 02:02:01:503.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Maximum session endpoints : 5 2025-09-01 Mon 02:02:01:503.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Fabric SNAT hash table size : 4096 2025-09-01 Mon 02:02:01:503.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Flow excluding Router ID in hash :0 2025-09-01 Mon 02:02:01:503.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Agent Mode : Vrouter 2025-09-01 Mon 02:02:01:503.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Gateway Mode : None 2025-09-01 Mon 02:02:01:503.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: DHCP Relay Mode : 0 2025-09-01 Mon 02:02:01:503.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Service instance netns cmd : /usr/local/bin/opencontrail-vrouter-netns 2025-09-01 Mon 02:02:01:503.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Service instance docker cmd : /usr/local/bin/opencontrail-vrouter-docker 2025-09-01 Mon 02:02:01:503.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Service instance workers : 0 2025-09-01 Mon 02:02:01:503.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Service instance timeout : 0 2025-09-01 Mon 02:02:01:503.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Service instance lb ssl : 2025-09-01 Mon 02:02:01:503.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Service instance lbaas auth : 2025-09-01 Mon 02:02:01:503.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Bgp as a service port range : 50000-50512 2025-09-01 Mon 02:02:01:503.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Services queue limit : 1024 2025-09-01 Mon 02:02:01:503.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: BGPAAS max shared sessions for service port : 4 2025-09-01 Mon 02:02:01:503.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Sandesh Key file : /etc/contrail/ssl/private/server-privkey.pem 2025-09-01 Mon 02:02:01:503.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Sandesh Cert file : /etc/contrail/ssl/certs/server.pem 2025-09-01 Mon 02:02:01:503.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Sandesh CA Cert : /etc/contrail/ssl/certs/ca-cert.pem 2025-09-01 Mon 02:02:01:503.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Sandesh SSL Enable : 0 2025-09-01 Mon 02:02:01:503.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Introspect SSL Enable : 0 2025-09-01 Mon 02:02:01:503.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Hypervisor mode : kvm 2025-09-01 Mon 02:02:01:504.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Ethernet Port Encap Type : 2025-09-01 Mon 02:02:01:504.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: Platform mode : Vrouter on host linux kernel 2025-09-01 Mon 02:02:01:504.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: VRouter [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-09-01 Mon 02:02:01:504.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Logging: DISABLED -> ENABLED 2025-09-01 Mon 02:02:01:504.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-09-01 Mon 02:02:01:521.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> self 2025-09-01 Mon 02:02:01:521.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:01:521.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:02:01:521.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-access-control-list -> self 2025-09-01 Mon 02:02:01:521.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap access-control-list : self -> virtual-network-access-control-list 2025-09-01 Mon 02:02:01:521.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap access-control-list : virtual-network-access-control-list -> nil 2025-09-01 Mon 02:02:01:521.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> self 2025-09-01 Mon 02:02:01:521.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:01:521.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> virtual-network-network-ipam 2025-09-01 Mon 02:02:01:521.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:01:521.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-qos-config -> self 2025-09-01 Mon 02:02:01:521.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-config : self -> virtual-network-qos-config 2025-09-01 Mon 02:02:01:521.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-config : virtual-network-qos-config -> nil 2025-09-01 Mon 02:02:01:521.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-security-logging-object -> self 2025-09-01 Mon 02:02:01:521.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : self -> virtual-network-security-logging-object 2025-09-01 Mon 02:02:01:521.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : virtual-network-security-logging-object -> nil 2025-09-01 Mon 02:02:01:521.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : logical-router-virtual-network -> self 2025-09-01 Mon 02:02:01:521.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-router-virtual-network : self -> logical-router-virtual-network 2025-09-01 Mon 02:02:01:521.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-router-virtual-network : logical-router-virtual-network -> logical-router-virtual-network 2025-09-01 Mon 02:02:01:521.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-router : self -> logical-router-virtual-network 2025-09-01 Mon 02:02:01:521.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-router : logical-router-virtual-network -> nil 2025-09-01 Mon 02:02:01:521.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:01:521.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-multicast-policy -> self 2025-09-01 Mon 02:02:01:521.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap multicast-policy : self -> virtual-network-multicast-policy 2025-09-01 Mon 02:02:01:521.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap multicast-policy : virtual-network-multicast-policy -> nil 2025-09-01 Mon 02:02:01:521.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for routing-instance 2025-09-01 Mon 02:02:01:521.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> self 2025-09-01 Mon 02:02:01:521.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:01:521.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-routing-instance 2025-09-01 Mon 02:02:01:521.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:02:01:521.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:02:01:521.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for security-group 2025-09-01 Mon 02:02:01:521.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-group : security-group-access-control-list -> self 2025-09-01 Mon 02:02:01:521.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap access-control-list : self -> security-group-access-control-list 2025-09-01 Mon 02:02:01:521.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap access-control-list : security-group-access-control-list -> nil 2025-09-01 Mon 02:02:01:521.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for tag 2025-09-01 Mon 02:02:01:521.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : application-policy-set-tag -> self 2025-09-01 Mon 02:02:01:521.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:01:521.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : policy-management-application-policy-set -> application-policy-set-tag 2025-09-01 Mon 02:02:01:521.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap policy-management : policy-management-application-policy-set -> nil 2025-09-01 Mon 02:02:01:521.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:02:01:521.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:521.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:521.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:01:521.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:01:521.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:02:01:521.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:01:521.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine : virtual-machine-interface-virtual-machine -> nil 2025-09-01 Mon 02:02:01:521.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:02:01:521.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:02:01:521.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> nil 2025-09-01 Mon 02:02:01:521.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:02:01:521.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:521.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:521.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:02:01:521.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:02:01:521.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-routing-instance -> self 2025-09-01 Mon 02:02:01:521.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface-routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:02:01:521.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface-routing-instance : virtual-machine-interface-routing-instance -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:02:01:521.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:02:01:521.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : virtual-machine-interface-routing-instance -> nil 2025-09-01 Mon 02:02:01:521.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-group -> self 2025-09-01 Mon 02:02:01:521.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-group : self -> virtual-machine-interface-security-group 2025-09-01 Mon 02:02:01:521.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-group : virtual-machine-interface-security-group -> nil 2025-09-01 Mon 02:02:01:521.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : alias-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap alias-ip : self -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap alias-ip : alias-ip-pool-alias-ip -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap alias-ip-pool : virtual-network-alias-ip-pool -> alias-ip-pool-alias-ip 2025-09-01 Mon 02:02:01:521.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-alias-ip-pool 2025-09-01 Mon 02:02:01:521.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-alias-ip-pool -> nil 2025-09-01 Mon 02:02:01:521.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:02:01:521.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:01:521.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:01:521.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:02:01:521.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:02:01:521.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:02:01:521.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:01:521.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:01:521.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:01:521.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:02:01:521.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap floating-ip : instance-ip-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap instance-ip : instance-ip-virtual-network -> instance-ip-floating-ip 2025-09-01 Mon 02:02:01:521.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> instance-ip-virtual-network 2025-09-01 Mon 02:02:01:521.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : instance-ip-virtual-network -> nil 2025-09-01 Mon 02:02:01:521.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : instance-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap instance-ip : self -> instance-ip-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap instance-ip : instance-ip-virtual-machine-interface -> nil 2025-09-01 Mon 02:02:01:521.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-route-table -> self 2025-09-01 Mon 02:02:01:521.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap interface-route-table : self -> virtual-machine-interface-route-table 2025-09-01 Mon 02:02:01:521.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap interface-route-table : virtual-machine-interface-route-table -> nil 2025-09-01 Mon 02:02:01:521.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : subnet-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap subnet : self -> subnet-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap subnet : subnet-virtual-machine-interface -> nil 2025-09-01 Mon 02:02:01:521.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:02:01:521.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:01:521.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:01:521.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:521.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-port-group-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:521.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> virtual-port-group-virtual-machine-interface 2025-09-01 Mon 02:02:01:521.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : physical-router-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:01:521.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:01:522.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:01:522.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:522.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-as-a-service : bgpaas-bgp-router -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-router : self -> bgpaas-bgp-router 2025-09-01 Mon 02:02:01:522.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-router : instance-bgp-router -> bgpaas-bgp-router 2025-09-01 Mon 02:02:01:522.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : self -> instance-bgp-router 2025-09-01 Mon 02:02:01:522.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : instance-bgp-router -> nil 2025-09-01 Mon 02:02:01:522.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:522.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-as-a-service : bgpaas-health-check -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap service-health-check : self -> bgpaas-health-check 2025-09-01 Mon 02:02:01:522.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap service-health-check : bgpaas-health-check -> nil 2025-09-01 Mon 02:02:01:522.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-qos-config -> self 2025-09-01 Mon 02:02:01:522.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-config : self -> virtual-machine-interface-qos-config 2025-09-01 Mon 02:02:01:522.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-config : virtual-machine-interface-qos-config -> nil 2025-09-01 Mon 02:02:01:522.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-logging-object -> self 2025-09-01 Mon 02:02:01:522.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : self -> virtual-machine-interface-security-logging-object 2025-09-01 Mon 02:02:01:522.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : virtual-machine-interface-security-logging-object -> nil 2025-09-01 Mon 02:02:01:522.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:02:01:522.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:01:522.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:02:01:522.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:02:01:522.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:01:522.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bridge-domain : virtual-machine-interface-bridge-domain -> nil 2025-09-01 Mon 02:02:01:522.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-tag -> self 2025-09-01 Mon 02:02:01:522.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> virtual-machine-interface-tag 2025-09-01 Mon 02:02:01:522.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-interface-tag 2025-09-01 Mon 02:02:01:522.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:01:522.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:02:01:522.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:522.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-tag -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:522.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> virtual-network-tag 2025-09-01 Mon 02:02:01:522.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : application-policy-set-tag -> virtual-network-tag 2025-09-01 Mon 02:02:01:522.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:01:522.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:02:01:522.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:01:522.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:01:522.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:02:01:522.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:02:01:522.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:01:522.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:02:01:522.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:02:01:522.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:02:01:522.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:01:522.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:01:522.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:02:01:522.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:02:01:522.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:01:522.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : project-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:522.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap project : self -> project-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap project : project-tag -> project-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> project-tag 2025-09-01 Mon 02:02:01:522.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : application-policy-set-tag -> project-tag 2025-09-01 Mon 02:02:01:522.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:01:522.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:522.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:01:522.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> self 2025-09-01 Mon 02:02:01:522.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> nil 2025-09-01 Mon 02:02:01:522.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:01:522.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : logical-router-interface -> self 2025-09-01 Mon 02:02:01:522.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-router : self -> logical-router-interface 2025-09-01 Mon 02:02:01:522.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-router : logical-router-interface -> nil 2025-09-01 Mon 02:02:01:522.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:02:01:522.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : physical-router-physical-interface -> self 2025-09-01 Mon 02:02:01:522.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:01:522.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:01:522.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:02:01:522.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : virtual-port-group-physical-interface -> self 2025-09-01 Mon 02:02:01:522.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-port-group-physical-interface : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:01:522.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-port-group-physical-interface : virtual-port-group-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:01:522.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-port-group : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:01:522.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> nil 2025-09-01 Mon 02:02:01:522.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:02:01:522.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> self 2025-09-01 Mon 02:02:01:522.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : self -> physical-interface-logical-interface 2025-09-01 Mon 02:02:01:522.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:02:01:522.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:01:522.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:01:522.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:02:01:522.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:02:01:522.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:02:01:522.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> nil 2025-09-01 Mon 02:02:01:522.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:02:01:522.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap logical-interface : physical-router-logical-interface -> self 2025-09-01 Mon 02:02:01:522.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : self -> physical-router-logical-interface 2025-09-01 Mon 02:02:01:522.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : physical-router-logical-interface -> nil 2025-09-01 Mon 02:02:01:522.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for physical-router 2025-09-01 Mon 02:02:01:522.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-router : physical-router-physical-interface -> self 2025-09-01 Mon 02:02:01:522.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:01:522.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap physical-interface : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:01:522.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:02:01:522.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:02:01:522.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:02:01:522.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> service-port-health-check 2025-09-01 Mon 02:02:01:522.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:522.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:01:522.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:01:522.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:02:01:522.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:02:01:522.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:02:01:522.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> service-port-health-check 2025-09-01 Mon 02:02:01:522.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> port-tuple-interface 2025-09-01 Mon 02:02:01:522.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:01:522.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:01:522.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:01:522.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for qos-config 2025-09-01 Mon 02:02:01:522.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-config : global-qos-config-qos-config -> self 2025-09-01 Mon 02:02:01:522.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap global-qos-config : global-qos-config-qos-config -> nil 2025-09-01 Mon 02:02:01:522.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for forwarding-class 2025-09-01 Mon 02:02:01:522.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap forwarding-class : forwarding-class-qos-queue -> self 2025-09-01 Mon 02:02:01:522.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-queue : self -> forwarding-class-qos-queue 2025-09-01 Mon 02:02:01:522.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap qos-queue : forwarding-class-qos-queue -> nil 2025-09-01 Mon 02:02:01:522.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:01:522.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : security-logging-object-network-policy -> self 2025-09-01 Mon 02:02:01:522.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap network-policy : self -> security-logging-object-network-policy 2025-09-01 Mon 02:02:01:522.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap network-policy : security-logging-object-network-policy -> nil 2025-09-01 Mon 02:02:01:522.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:01:522.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : security-logging-object-security-group -> self 2025-09-01 Mon 02:02:01:522.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-group : self -> security-logging-object-security-group 2025-09-01 Mon 02:02:01:523.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-group : security-logging-object-security-group -> nil 2025-09-01 Mon 02:02:01:523.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:01:523.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : firewall-policy-security-logging-object -> self 2025-09-01 Mon 02:02:01:523.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-security-logging-object : self -> firewall-policy-security-logging-object 2025-09-01 Mon 02:02:01:523.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-security-logging-object : firewall-policy-security-logging-object -> firewall-policy-security-logging-object 2025-09-01 Mon 02:02:01:523.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : firewall-policy-security-logging-object -> nil 2025-09-01 Mon 02:02:01:523.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:01:523.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap security-logging-object : firewall-rule-security-logging-object -> self 2025-09-01 Mon 02:02:01:523.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule-security-logging-object : self -> firewall-rule-security-logging-object 2025-09-01 Mon 02:02:01:523.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule-security-logging-object : firewall-rule-security-logging-object -> firewall-rule-security-logging-object 2025-09-01 Mon 02:02:01:523.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : firewall-rule-security-logging-object -> nil 2025-09-01 Mon 02:02:01:523.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for virtual-router 2025-09-01 Mon 02:02:01:523.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-router : virtual-router-sub-cluster -> self 2025-09-01 Mon 02:02:01:523.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap sub-cluster : self -> virtual-router-sub-cluster 2025-09-01 Mon 02:02:01:523.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap sub-cluster : bgp-router-sub-cluster -> virtual-router-sub-cluster 2025-09-01 Mon 02:02:01:523.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-router : self -> bgp-router-sub-cluster 2025-09-01 Mon 02:02:01:523.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bgp-router : bgp-router-sub-cluster -> nil 2025-09-01 Mon 02:02:01:523.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:02:01:523.115 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:02:01:523.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:01:523.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:01:523.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:01:523.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:02:01:523.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:02:01:523.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:02:01:523.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:01:523.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:02:01:523.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for application-policy-set 2025-09-01 Mon 02:02:01:523.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> self 2025-09-01 Mon 02:02:01:523.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:523.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:523.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:01:523.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:01:523.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:01:523.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:01:523.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : firewall-rule-service-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap service-group : self -> firewall-rule-service-group 2025-09-01 Mon 02:02:01:523.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap service-group : firewall-rule-service-group -> nil 2025-09-01 Mon 02:02:01:523.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:01:523.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:01:523.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : firewall-rule-address-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap address-group : self -> firewall-rule-address-group 2025-09-01 Mon 02:02:01:523.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap address-group : address-group-tag -> firewall-rule-address-group 2025-09-01 Mon 02:02:01:523.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> address-group-tag 2025-09-01 Mon 02:02:01:523.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : address-group-tag -> nil 2025-09-01 Mon 02:02:01:523.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:01:523.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:01:523.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : firewall-rule-tag -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : self -> firewall-rule-tag 2025-09-01 Mon 02:02:01:523.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap tag : firewall-rule-tag -> nil 2025-09-01 Mon 02:02:01:523.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:01:523.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:01:523.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:01:523.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Adding ReactorMap firewall-rule : firewall-policy-firewall-rule -> nil 2025-09-01 Mon 02:02:01:674.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.18:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:01:679.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Current receive sock buffer size is 262144 2025-09-01 Mon 02:02:01:679.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390954796544, Pid 24715]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU 2025-09-01 Mon 02:02:01:682.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Vrouter family is 34 2025-09-01 Mon 02:02:01:683.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Flow table size : 20185088 2025-09-01 Mon 02:02:01:683.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Flow table size : 161218560 2025-09-01 Mon 02:02:01:683.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SetHugePages: Bridge table size:20185088 Flow table size:161218560 2025-09-01 Mon 02:02:01:683.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Mem mapping hugepage file:/dev/hugepages/bridge size:20185088 2025-09-01 Mon 02:02:01:683.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Mem mapped hugepage file:/dev/hugepages/bridge to addr:0x7faf4ee00000 2025-09-01 Mon 02:02:01:683.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Mem mapping hugepage file:/dev/hugepages/flow size:161218560 2025-09-01 Mon 02:02:01:683.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Mem mapped hugepage file:/dev/hugepages/flow to addr:0x7faf45400000 2025-09-01 Mon 02:02:01:683.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Sending Huge Page configuration to VROUTER 2025-09-01 Mon 02:02:01:742.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Huge pages set successfully 2025-09-01 Mon 02:02:01:763.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Flow table size : 161218560 2025-09-01 Mon 02:02:01:763.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Flow table size : 20185088 2025-09-01 Mon 02:02:01:824.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: ROLE : Generator 2025-09-01 Mon 02:02:01:824.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: MODULE : contrail-vrouter-agent 2025-09-01 Mon 02:02:01:824.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: SOURCE : an-jenkins-deploy-platform-ansible-os-3995-1 2025-09-01 Mon 02:02:01:824.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: NODE TYPE : Compute 2025-09-01 Mon 02:02:01:824.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: INSTANCE ID : 0 2025-09-01 Mon 02:02:01:824.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: HTTP SERVER PORT : 8085 2025-09-01 Mon 02:02:01:824.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-09-01 Mon 02:02:01:825.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: HTTP Introspect Init 2025-09-01 Mon 02:02:01:825.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8085 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:01:825.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Sandesh Http Server Port: 8085 2025-09-01 Mon 02:02:01:826.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: NOT Writing http_port 8085TO : /tmp/contrail-vrouter-agent.24041.http_port 2025-09-01 Mon 02:02:01:826.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: CONNECT TO COLLECTOR: 1 2025-09-01 Mon 02:02:01:826.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Idle 2025-09-01 Mon 02:02:01:828.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: No Client: 1756692121827547 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:01:829.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvStart in state Idle 2025-09-01 Mon 02:02:01:829.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Disconnect 2025-09-01 Mon 02:02:01:829.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Disconnect 2025-09-01 Mon 02:02:01:829.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692121829082 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:01:829.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Disconnect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:829.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Disconnect 2025-09-01 Mon 02:02:01:829.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692121829169 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:01:829.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Disconnect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:01:830.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvCollectorUpdate in state Disconnect 2025-09-01 Mon 02:02:01:830.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:02:01:830.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:01:830.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121830321 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:01:830.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:01:830.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:01:830.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121830347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:01:830.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:830.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:01:830.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121830625 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:01:830.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:01:830.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:01:830.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:01:830.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:01:831.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:01:831.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:01:831.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:02:01:831.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:831.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121831052 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:01:831.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:01:831.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:831.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121831071 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:01:831.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:831.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:01:832.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:832.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121831182 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:01:832.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:01:833.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:835.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692121833005 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:02:01:835.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:02:01:835.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:835.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692121833740 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:02:01:835.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:02:01:835.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface lo flags 65609 2025-09-01 Mon 02:02:01:835.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface ens3 flags 69699 2025-09-01 Mon 02:02:01:835.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface ens6 flags 69699 2025-09-01 Mon 02:02:01:835.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface docker0 flags 4099 2025-09-01 Mon 02:02:01:835.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface pkt1 flags 65601 2025-09-01 Mon 02:02:01:835.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface pkt3 flags 65601 2025-09-01 Mon 02:02:01:835.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface pkt2 flags 65601 2025-09-01 Mon 02:02:01:835.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface vhost0 flags 69699 2025-09-01 Mon 02:02:01:835.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink interface message RTM_NEWLINK for interface pkt0 flags 69699 2025-09-01 Mon 02:02:01:835.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:02:01:835.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:02:01:835.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface docker0 2025-09-01 Mon 02:02:01:835.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:02:01:835.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:02:01:835.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:02:01:835.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:02:01:835.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink address message RTM_NEWADDR for interface pkt0 2025-09-01 Mon 02:02:01:835.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:02:01:835.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 10.0.0.0/24 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 10.0.0.1/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 10.0.0.2/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 10.0.0.4/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:02:01:836.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Handle netlink route message RTM_NEWROUTE : 172.17.0.0/16 Interface docker0 GW 0.0.0.0 2025-09-01 Mon 02:02:01:836.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:01:836.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:01:836.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:01:836.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:01:836.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:01:836.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:01:836.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:01:836.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:01:836.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:01:836.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:01:836.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:01:836.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:01:836.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:01:836.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:02:01:836.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:02:01:836.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:02:01:836.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:01:836.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:01:836.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:01:836.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:01:856.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:9091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:01:856.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Router ID configured. Connection to controller initiated 2025-09-01 Mon 02:02:01:860.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:860.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121860422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:860.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:860.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:860.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121860741 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, Collector connection down, No Configuration for self Number of connections:2, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:01:860.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:861.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:861.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692121861368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:861.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:861.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:861.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692121861836 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, Collector connection down, No Configuration for self Number of connections:3, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:01:861.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:862.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:862.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692121862254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:862.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:862.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:862.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692121862753 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:01:862.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:863.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:863.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692121863340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:863.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:863.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:863.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692121863775 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:01:863.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:01:865.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server [fe80::f816:3eff:fe37:b899%vhost0]:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:01:910.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:910.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692121910584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:910.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:910.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:910.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692121910851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:910.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:911.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:911.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692121911204 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:911.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:911.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:911.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692121911328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38341 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:911.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:911.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:911.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121911440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:911.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:911.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:911.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121911522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:911.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:911.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:911.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121911528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:911.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:911.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:911.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692121911614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38493 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:911.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:912.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:01:913.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:01:913.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:01:913.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:01:913.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:01:913.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:913.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121913624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:913.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:913.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:01:913.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:01:913.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:913.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121913759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:913.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:913.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:913.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121913811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:913.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:01:913.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:01:914.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:01:914.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692121913902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:01:914.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:831.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:02:831.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:02:02:831.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:02:831.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692122831338 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:02:831.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:831.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:02:831.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692122831773 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 12, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:02:831.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:02:833.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:02:833.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:02:833.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:02:833.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:02:833.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:833.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Idle 2025-09-01 Mon 02:02:02:833.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:833.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692122833420 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 12, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:02:833.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:02:833.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:833.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692122833449 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:02:833.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:833.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:02:833.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:833.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692122833588 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 13, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:02:833.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:02:876.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:877.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692122876915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:877.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:877.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:877.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692122877450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44679 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:877.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:877.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:878.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:878.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:878.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692122878205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:878.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:940.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:940.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692122939760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:940.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:940.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:940.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692122940312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41319 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:940.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:940.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:940.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692122940330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:940.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:940.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:940.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:940.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692122940447 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:941.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:941.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:941.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:941.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:941.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:941.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692122940503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:941.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:941.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:941.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692122940644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:941.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:941.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:941.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692122941160 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:941.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:941.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:941.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:941.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692122941782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:941.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:941.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:941.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:941.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692122941974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:941.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:760.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:760.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692124759927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:760.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:761.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:761.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692124761219 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33411 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:761.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:761.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:761.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:761.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:761.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692124761782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:761.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:778.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:778.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692124778071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:778.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:778.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:778.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692124778505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:778.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:778.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:778.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:778.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:778.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692124778862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:778.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:833.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:04:835.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:02:04:835.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:04:835.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692124833765 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:04:835.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:04:835.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:04:835.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692124835539 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 25, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 53, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:04:835.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:04:836.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:04:836.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:04:836.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:04:836.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:04:837.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:837.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Idle 2025-09-01 Mon 02:02:04:837.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:837.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692124836566 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 25, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 55, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:04:837.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:04:837.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:837.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692124836601 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:04:837.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:04:837.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:04:838.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:838.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692124837660 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 26, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 55, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:04:838.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:04:903.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:903.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692124903273 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:903.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:904.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:904.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692124904087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:904.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:904.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:904.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:904.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:904.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692124904373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:904.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:948.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:948.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692124948329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:948.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:948.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:948.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692124948704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:948.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:948.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:949.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:949.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:949.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692124949091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:949.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:595.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:595.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128595089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:595.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:595.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:595.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692128595464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42531 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:595.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:595.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:08:595.854 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:595.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:596.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128595929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:596.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:596.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:596.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128595949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:596.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:596.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:596.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692128596130 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:596.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:597.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:597.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692128597185 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:08:597.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:598.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:598.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:08:598.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128598127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:598.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:598.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:598.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128598172 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:598.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:612.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:612.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128612503 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:612.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:612.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:612.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128612686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:612.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:612.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:08:612.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:613.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:613.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128613018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:613.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:613.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:613.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128613023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:613.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:613.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:613.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128613129 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:613.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:613.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:613.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692128613891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:613.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:613.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:614.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692128613916 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:08:614.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:614.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:614.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692128614637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:614.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:614.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:614.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692128614678 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:614.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:614.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:08:646.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:646.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692128646375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:646.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:646.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:647.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692128646938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39609 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:647.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:647.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:08:647.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:647.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:647.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128647297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:647.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:662.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:662.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128662846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:662.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:663.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:663.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692128663028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:663.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:663.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:08:663.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:663.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:663.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128663434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:663.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:794.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:794.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692128794060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:794.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:794.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:794.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692128794736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:794.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:795.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:08:795.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:795.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:795.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128795230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:795.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:795.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:795.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128795240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:795.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:795.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:795.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128795355 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:795.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:795.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:795.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128795413 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:08:795.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:796.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:796.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128795992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:796.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:796.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:796.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128796051 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:796.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:796.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:08:837.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:08:838.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:02:08:838.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:08:838.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128837825 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:838.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:838.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:08:838.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128838210 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 43, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 95, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:08:838.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:08:838.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:08:838.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:08:838.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:08:838.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:08:838.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:838.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Idle 2025-09-01 Mon 02:02:08:838.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:839.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692128838734 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 43, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 97, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:08:839.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:08:839.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:839.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692128838791 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:839.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:839.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:08:839.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:839.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692128838912 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 44, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 97, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:08:839.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:08:844.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:844.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128844151 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:844.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:844.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:844.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692128844332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:844.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:08:844.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:08:844.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:844.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:844.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692128844725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:08:844.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:161.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:161.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692129161323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:161.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:162.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:162.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692129162013 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:162.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:162.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:162.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:162.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:162.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692129162483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:162.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:162.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:162.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692129162490 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:162.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:162.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:162.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692129162559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:162.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:162.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:162.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692129162672 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:162.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:162.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:162.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692129162699 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:09:162.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:163.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:163.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692129163358 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:163.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:163.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:163.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692129163413 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:163.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:163.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:09:208.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:208.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692129208646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:208.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:209.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:209.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692129209309 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38631 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:209.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:209.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:209.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:209.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:209.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692129209694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:209.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:592.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:592.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692129592466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:592.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:593.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:593.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692129593059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:593.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:593.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:593.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:593.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:593.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692129593620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:593.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:680.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:680.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692129680610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:680.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:681.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:681.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692129681194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42121 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:681.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:681.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:681.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:681.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:681.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692129681835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:681.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:861.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:861.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692129861837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:861.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:862.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:862.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692129862424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:862.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:862.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:862.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:862.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:862.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692129862861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:862.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:226.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:226.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692130226890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:227.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:227.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:227.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692130227446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:227.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:227.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:10:227.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:10:227.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:227.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692130227835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:227.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:591.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:591.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692131591793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:591.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:592.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:592.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692131592279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:592.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:592.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:11:592.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:11:592.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:592.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692131592722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:592.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:735.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:735.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692131735036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:735.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:735.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:735.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692131735254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42529 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:735.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:735.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:11:735.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:11:735.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:735.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692131735679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:735.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:770.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:770.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692131770021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:770.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:770.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:770.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692131770648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:770.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:771.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:11:771.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:11:771.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:771.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692131771210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:771.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:046.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:046.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692132045996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:046.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:046.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:046.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692132046518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38705 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:046.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:046.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:12:046.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:12:046.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:046.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692132046862 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:046.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:633.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:41150 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:15:336.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:337.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692135336922 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:337.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:337.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:337.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135337559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39425 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:337.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:337.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:337.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:337.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:337.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692135337897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:337.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:337.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:337.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692135337905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:337.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:338.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:338.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692135337997 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:338.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:338.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:338.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692135338048 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:15:338.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:338.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:338.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135338636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:338.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:338.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:338.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:15:338.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135338689 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:338.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:384.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:384.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135384842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:384.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:385.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:385.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135385037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:385.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:385.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:385.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:385.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:385.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135385551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:385.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:899.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:899.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:900.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692135899031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:900.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:900.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:900.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692135899535 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:900.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:901.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:901.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:901.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135901063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:901.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:901.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:901.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135901069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:901.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:901.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:901.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135901132 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:901.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:901.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:901.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135901223 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:901.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:901.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:901.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135901244 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:15:901.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:902.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:902.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135901957 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:902.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:902.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:15:902.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:902.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135901998 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:902.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:947.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:947.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692135947781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:947.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135948381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:948.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135948424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:948.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135948543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:948.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:948.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:948.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:948.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692135948737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:948.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692135948741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:948.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:948.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135948840 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:948.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:948.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:948.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135948854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:948.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:948.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:949.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135948890 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:15:949.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:949.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:949.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135949411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:949.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:949.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:949.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:15:949.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135949454 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:949.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:983.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:983.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135983290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:983.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:983.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:983.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692135983637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:983.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:983.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:984.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:984.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:984.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135984164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:984.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:984.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:984.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135984171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:984.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:984.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:984.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135984217 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:984.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:984.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:984.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135984302 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:984.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:984.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:984.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692135984324 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:15:984.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:984.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:984.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135984861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:984.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:15:984.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:985.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:985.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692135984899 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:15:985.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:15:995.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:995.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135995564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:995.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:995.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:995.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135995680 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44191 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:995.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:15:995.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:15:995.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:15:996.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:15:996.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692135996023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:15:996.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:032.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:032.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692136032069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:032.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:032.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:032.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692136032574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:032.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:032.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:032.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:032.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:032.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692136032899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:032.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:330.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:330.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692136330743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:330.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:331.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:331.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692136331141 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:331.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:331.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:331.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:331.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:331.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692136331786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:331.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:838.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:16:839.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:02:16:839.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:16:839.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692136839087 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:16:839.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:839.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:16:839.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692136839752 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 131 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 188, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:16:839.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:16:840.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:16:840.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:16:840.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:16:840.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:16:840.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:840.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Idle 2025-09-01 Mon 02:02:16:840.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:840.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692136840201 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 131 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 190, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:16:840.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:16:840.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:840.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692136840244 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:16:840.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:840.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:16:840.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:840.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692136840340 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 43 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 131 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 91, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 190, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:16:840.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:16:894.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:894.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692136893996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:894.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:894.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:894.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692136894695 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43141 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:894.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:895.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:895.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:895.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:895.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692136895258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:895.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:969.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:969.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692136969078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:969.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:969.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:969.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692136969856 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:969.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:970.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:970.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:970.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:970.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692136970092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:970.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:058.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:058.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692137058225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:058.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:058.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:058.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692137058882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34925 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:058.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:059.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:17:059.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:17:059.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:059.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692137059163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:059.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:276.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:276.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692138275981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:276.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:276.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:276.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692138276631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36363 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:276.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:277.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:277.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:277.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:277.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692138277390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:277.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:913.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:913.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:913.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692138913297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:913.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:913.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:913.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:913.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692138913469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37181 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:913.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:913.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:913.969 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692138913793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:913.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:914.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:914.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692138914152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:914.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:914.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:914.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692138914531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40603 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:914.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:914.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:914.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:914.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:914.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692138914703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:914.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:965.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:965.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692138965382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:965.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:965.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:965.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692138965787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35719 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:965.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:966.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:966.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:966.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:966.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692138966201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:966.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:949.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:949.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692141949600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:949.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:950.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:950.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692141950164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:950.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:950.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:21:950.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:21:950.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:951.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692141950971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:951.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:951.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:951.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692141950976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:951.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:951.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:951.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692141951035 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:21:951.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:21:951.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:951.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692141951086 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:21:951.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:21:951.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:951.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692141951622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:951.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:951.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:951.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692141951679 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:21:951.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:21:951.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:22:003.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:003.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692142003125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:003.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:003.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:003.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692142003674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:003.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:003.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:22:004.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:22:004.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:004.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692142004062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:004.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:766.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:766.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692142766847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:766.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:767.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:767.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692142767430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:767.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:767.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:22:767.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:22:767.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:767.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692142767776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:767.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:767.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:767.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692142767783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:767.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:767.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:767.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692142767855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:22:767.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:22:767.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:768.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692142767973 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:22:768.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:22:768.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:768.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692142768006 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:22:768.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:22:768.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:768.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692142768598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:768.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:768.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:768.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:22:768.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692142768640 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:22:768.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:22:820.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:820.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692142820810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:820.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:821.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:821.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692142821283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39615 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:821.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:22:821.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:22:821.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:22:821.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:22:821.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692142821778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:22:821.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:012.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:012.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692143012210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:012.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:012.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:012.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143012840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33417 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:012.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:013.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:013.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:013.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:013.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692143013317 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:013.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:053.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:054.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:054.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:054.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143053904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:054.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:054.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:054.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143054091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:054.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:054.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:054.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143054455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:054.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:054.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:054.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143054460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:054.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:054.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:055.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143054535 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:055.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:055.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:055.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143054603 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:23:055.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:055.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:055.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692143055316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:055.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:055.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:055.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692143055380 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:055.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:055.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:23:102.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:102.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692143102486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:102.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:103.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:103.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143103055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:103.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:103.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:103.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:103.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:103.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692143103576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:103.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:250.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:250.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692143250382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:250.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:251.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:251.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692143251010 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38637 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:251.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:251.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:251.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:251.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:251.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143251446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:251.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:251.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:251.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692143251455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:251.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:251.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:251.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692143251570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:251.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:251.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:251.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692143251711 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:251.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:251.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:251.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692143251738 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:23:251.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:252.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:252.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692143252804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:252.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:252.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:252.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692143252865 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:252.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:252.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:23:302.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:302.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692143302097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:302.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:302.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:302.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692143302288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43519 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:302.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:302.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:302.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:302.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:302.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692143302674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:302.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:802.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:803.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692143802943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:803.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:803.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:803.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692143803620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:803.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:804.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:804.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:804.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:804.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692143804147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:804.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:156.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:156.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692144156673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:156.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:157.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:157.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692144157299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:157.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:157.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:157.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:157.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:157.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692144157729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:157.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:364.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:364.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692144364822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:364.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:365.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:365.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:365.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:366.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692144365500 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46823 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:366.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:366.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:366.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692144365814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:366.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:993.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:993.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692144993454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:993.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:993.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:993.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692144993664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:993.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:993.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:994.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:994.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:994.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692144994143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:994.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:622.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:622.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692145622304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:622.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:622.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:622.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692145622917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:623.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:623.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:623.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:623.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:623.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692145623367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:623.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:101.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:103.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692146101898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:103.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:102.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:26:103.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:103.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692146102191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40687 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:103.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:103.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:103.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:103.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692146103945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:103.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:490.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:490.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692146490009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:490.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:490.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:490.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692146490825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37563 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:490.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:491.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:26:491.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:491.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:491.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692146491211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:491.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:098.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:098.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692149098283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:098.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:098.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:099.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149098944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:099.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:099.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:29:099.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:29:099.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:099.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149099384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:099.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:099.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:099.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149099392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:099.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:099.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:099.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149099494 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:099.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:099.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:099.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149099559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:29:099.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:100.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:100.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149100152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:100.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:100.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:100.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:29:100.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149100192 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:100.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:150.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:150.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692149150388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:150.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:150.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:150.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149150670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39235 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:150.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:150.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:29:150.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:29:151.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:151.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149151000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:151.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:619.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:619.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149619487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:619.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:619.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:619.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692149619708 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:619.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:620.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:29:620.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:29:620.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:620.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692149620251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:620.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:620.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:620.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692149620261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:620.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:620.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:620.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692149620364 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:620.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:620.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:620.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692149620473 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:620.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:620.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:620.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692149620495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:29:620.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:621.117 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:621.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149621085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:621.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:621.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:621.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149621136 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:621.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:29:621.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:674.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:674.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692149674268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:674.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:674.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:675.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149674946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34857 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:675.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:675.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:29:675.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:29:675.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:675.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692149675372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:675.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:848.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:848.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692149848096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:848.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:848.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:848.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692149848437 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39743 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:848.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:848.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:29:849.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:29:849.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:849.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149849090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:849.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:849.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:849.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149849101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:849.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:849.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:849.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149849190 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:849.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:849.658 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:849.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692149849249 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:29:849.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:850.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:850.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:29:850.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149850054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:850.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:850.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:850.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692149850099 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:29:850.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:29:899.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:899.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692149899271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:899.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:899.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:899.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692149899872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:899.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:29:900.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:29:900.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:29:900.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:29:900.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692149900364 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:29:900.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:105.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:105.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692150105143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:105.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:105.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:105.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150105761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40587 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:105.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:105.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:106.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:106.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:106.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692150106051 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:106.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:199.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:199.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692150199442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:199.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:199.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:199.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692150199709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36119 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:199.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:199.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:200.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:200.070 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:200.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150200059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:200.094 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:200.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:200.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150200090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:200.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:200.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:200.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150200180 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:30:200.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:200.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:200.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692150200436 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:30:200.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:200.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:200.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692150200462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:30:200.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:201.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:201.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150201126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:201.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:201.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:201.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:30:201.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150201172 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:30:201.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:251.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:251.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692150251421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:251.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:252.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:252.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692150252027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:252.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:252.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:252.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:252.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:252.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692150252656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:252.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:710.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:710.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692150710568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:710.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:711.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:711.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692150710987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:711.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:711.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:711.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:711.439 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:711.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692150711428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:711.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:944.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:944.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692150944581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:944.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:945.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:945.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692150945171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43665 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:945.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:945.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:946.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:946.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:946.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692150946041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:946.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:179.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:179.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692151179820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:179.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:180.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:180.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692151180520 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38149 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:180.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:180.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:31:181.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:31:181.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:181.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692151181074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:181.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:506.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:507.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692151506257 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [ [ [ name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ] 2025-09-01 Mon 02:02:31:507.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVirtualNetworkAgentTrace 2025-09-01 Mon 02:02:31:507.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:508.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692151507488 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 interface_list= [ [ ] ] error_intf_list= [ [ ] ] no_config_intf_list= [ [ (*_iter151) = 00000000-0000-0000-0000-000000000000, ] ] vmi_count= [ active = 1 ] down_interface_count = 1 ] 2025-09-01 Mon 02:02:31:508.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:02:31:828.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:828.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692151828306 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 344 dequeues = 344 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 360 dequeues = 360 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692136840304 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692151507703 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:02:31:828.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:31:828.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:828.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692151828511 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 72 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 344, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:31:828.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:31:836.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:836.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692151836598 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] self_ip_list= [ [ (*_iter146) = 10.20.0.18, ] ] build_info = {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} phy_if= [ [ [ name = ens6 mac_address = fa:16:3e:37:b8:99 ], ] ] vhost_if= [ name = vhost0 mac_address = fa:16:3e:37:b8:99 ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_DEBUG log_category = sandesh_http_port = 8085 dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.25, ] ] vhost_cfg= [ name = vhost0 ip = 10.20.0.18 ip_prefix_len = 24 gateway = 0.0.0.0 ] eth_name= [ [ (*_iter150) = ens6, ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.20.0.18 dns_server_list_cfg= [ [ (*_iter152) = 10.20.0.25:53, (*_iter152) = 10.20.0.193:53, (*_iter152) = 10.20.0.129:53, ] ] control_node_list_cfg= [ [ (*_iter153) = 10.20.0.25:5269, (*_iter153) = 10.20.0.193:5269, (*_iter153) = 10.20.0.129:5269, ] ] hostname_cfg = an-jenkins-deploy-platform-ansible-os-3995-1 flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 gateway_cfg_list= [ [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [ [ (*_iter155) = 10.0.0.249:8086, (*_iter155) = 10.0.0.38:8086, (*_iter155) = 10.0.0.131:8086, ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [ max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-09-01 00:11:15.602301", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-10717-1.novalocal"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name = vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [ [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [ [ (*_iter161) = 0.0.0.0, ] ] ] 2025-09-01 Mon 02:02:31:836.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:02:31:836.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:836.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692151836683 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 102 in_bytes = 7141 out_tpkts = 117 out_bytes = 6661 exception_packets = 19 exception_packets_dropped = 9 exception_packets_allowed = 10 total_flows = 0 aged_flows = 0 uptime = 1756692121504528 phy_if_info= [ [ _iter254->first = ens6 [ speed = -1 duplexity = 255 ], ] ] raw_vhost_stats= [ name = vhost0 in_pkts = 102 in_bytes = 7141 out_pkts = 117 out_bytes = 6661 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 101 in_bytes = 5989 out_pkts = 105 out_bytes = 7267 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:02:31:836.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:31:836.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:836.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692151836722 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:02:31:836.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:02:32:068.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:068.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692152068214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:068.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:068.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:068.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:068.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692152068507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39695 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:068.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:069.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:069.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:069.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692152069723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:069.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:655.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:656.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692152655660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:656.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:656.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:656.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692152655910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39031 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:656.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:656.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:656.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:656.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:656.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692152656418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:656.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:746.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:746.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692152746260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:746.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:746.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:746.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692152746871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:746.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:747.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:747.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:747.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:747.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692152747419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:747.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:840.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:32:840.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:02:32:840.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:32:840.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692152840495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:32:840.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:32:840.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:32:841.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692152840901 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 358, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:32:841.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:32:841.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:32:841.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:32:841.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:32:841.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:32:841.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:841.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Idle 2025-09-01 Mon 02:02:32:841.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:841.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692152841477 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 175, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 360, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:32:841.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:32:841.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:841.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692152841553 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:32:841.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:32:841.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:32:841.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:842.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692152841660 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 176, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 360, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:32:842.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:32:981.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:981.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692152981240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:981.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:981.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:981.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692152981467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:981.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:981.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:982.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:982.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:982.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692152982080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:982.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:850.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:850.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692155849956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:850.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:35:850.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:850.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:850.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692155850265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:850.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:850.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:35:850.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:850.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692155850754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:850.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:850.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:850.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692155850761 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:850.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:850.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:850.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692155850869 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:35:850.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:35:850.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:850.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692155850931 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:35:850.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:35:851.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:851.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692155851577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:851.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:851.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:851.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:35:851.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692155851623 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:35:851.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:35:898.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:898.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692155898818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:898.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:899.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:899.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692155899450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40115 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:899.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:35:899.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:35:899.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:35:899.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:35:899.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692155899910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:35:899.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:599.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:599.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692156599565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:599.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:600.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:600.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692156600222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33831 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:600.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:600.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:600.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:600.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:600.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156600647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:600.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:600.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:600.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156600653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:600.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:600.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:600.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156600753 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:600.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:600.821 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:600.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156600816 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:36:600.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:601.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:601.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156601417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:601.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:601.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:601.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156601453 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:601.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:601.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:36:650.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:650.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692156650588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:650.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:651.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:651.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692156651193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41025 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:651.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:651.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:651.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:651.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:651.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692156651502 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:651.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:724.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:724.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692156724565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:724.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:724.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:724.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692156724811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41079 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:724.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:725.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:725.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:725.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:725.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156725249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:725.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:725.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:725.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156725262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:725.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:725.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:725.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156725336 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:725.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:725.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:725.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692156725483 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:725.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:725.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:725.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692156725503 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:36:725.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:726.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:726.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692156726335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:726.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:726.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:726.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:36:726.425 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692156726379 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:726.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:775.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:775.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692156775519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:775.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:775.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:775.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692156775768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:775.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:775.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:776.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:776.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:776.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692156776079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:776.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:870.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:870.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692156870301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:870.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:870.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:870.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156870895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46281 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:870.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:871.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:871.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:872.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156871438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:872.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:872.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156871446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:872.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:872.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156871542 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:872.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:872.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156871644 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:872.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:872.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156871665 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:36:872.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:872.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156872300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:872.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:872.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:872.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692156872333 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:872.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:872.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:36:925.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:925.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692156925805 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:925.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:926.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:926.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692156926426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:926.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:926.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:927.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:927.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:927.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692156927227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:927.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:962.120 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:962.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692156962073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:962.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:962.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:962.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692156962304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:962.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:962.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:962.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:962.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:962.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692156962756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:962.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:686.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:686.878 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692157686625 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:686.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:688.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:688.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692157688003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45055 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:688.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:688.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:37:688.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:37:688.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:688.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692157688282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:688.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:820.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:820.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692157820190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:820.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:821.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:821.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692157821597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34141 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:821.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:821.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:37:821.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:37:821.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:821.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692157821880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:821.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:899.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:899.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692157899439 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:899.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:899.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:899.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692157899634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:899.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:900.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:37:900.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:37:900.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:900.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692157900193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:900.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:961.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:961.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692158960968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:961.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:961.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:961.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692158961599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:961.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:961.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:38:962.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:38:962.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:962.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692158962057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:962.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:614.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:614.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:39:614.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:39:614.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692159614391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:615.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:615.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:615.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692159614575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39521 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:615.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:615.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:615.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692159615026 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:615.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:676.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:676.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:39:676.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692159676067 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:676.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:676.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:676.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692159676283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45697 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:676.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:676.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:39:676.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:676.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692159676857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:676.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:024.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:024.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692160024375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:024.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:025.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:025.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692160025009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41673 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:025.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:025.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:40:025.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:025.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:025.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692160025415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:025.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:174.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:174.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163174261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:174.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:174.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:174.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163174464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:174.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:174.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:174.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:175.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:175.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163174992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:175.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:175.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:175.059 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163175028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:175.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:175.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:175.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:43:175.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163175126 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:175.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:175.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:175.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163175191 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:43:175.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:175.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:175.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163175765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:175.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:175.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:176.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163175824 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:176.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:224.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:225.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163224900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:225.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:225.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:225.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163225476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:225.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:225.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:225.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:225.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:225.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692163225916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:225.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:421.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:421.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163420969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:421.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:421.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:421.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692163421443 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43789 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:421.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:421.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:421.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:421.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:421.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163421797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:421.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:421.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:421.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163421804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:421.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:421.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:421.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163421882 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:421.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:421.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:422.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692163421978 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:422.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:422.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:422.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692163421997 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:43:422.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:422.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:422.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692163422584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:422.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:422.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:422.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692163422629 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:422.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:422.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:43:473.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:473.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163473795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:473.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:473.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:473.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692163473965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:473.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:474.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:474.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:474.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:474.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163474420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:474.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:791.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:791.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163791210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:791.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:791.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:791.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692163791413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:791.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:791.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:791.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:791.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:792.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:43:792.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163791731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:792.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:792.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:792.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163791748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:792.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:792.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:792.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163791804 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:792.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:792.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:792.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163791858 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:43:792.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:792.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:792.695 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163792429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:792.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:792.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:792.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692163792474 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:792.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:841.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:841.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163841582 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:841.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:842.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:842.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692163842057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39829 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:842.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:842.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:842.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:842.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:842.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692163842631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:842.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:985.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:985.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692163985583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:985.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:985.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:985.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692163985764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40823 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:985.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:986.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:986.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:986.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:986.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:986.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:986.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:986.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986267 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:986.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:986.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986350 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:986.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:986.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986369 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:43:986.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:986.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:986.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:986.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:986.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692163986970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:987.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:987.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:44:037.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:037.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692164037302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:037.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:037.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:037.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692164037834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34221 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:037.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:038.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:44:038.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:44:038.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:038.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692164038138 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:038.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:171.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:171.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692164171078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:171.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:171.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:171.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692164171290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33265 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:171.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:171.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:44:171.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:44:171.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:171.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692164171671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:171.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:482.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:482.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692164482571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:482.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:483.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:483.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692164483077 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40743 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:483.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:483.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:44:483.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:44:483.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:483.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692164483645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:483.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:850.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:850.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692164850766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:850.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:851.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:851.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692164851347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:851.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:851.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:44:852.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:44:852.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:852.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692164852036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:852.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:001.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:001.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692165001244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:001.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:001.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:001.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692165001694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39059 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:001.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:001.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:45:001.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:45:001.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:001.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692165001956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:001.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:079.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:079.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692166079829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:079.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:080.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:080.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692166080274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38613 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:080.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:080.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:080.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:080.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:080.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692166080757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:080.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:445.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:445.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692166445811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:445.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:446.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:446.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692166446354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:446.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:446.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:446.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:446.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:446.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692166446901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:446.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:940.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:940.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692166940195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:940.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:940.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:940.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692166940664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33115 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:940.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:941.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:941.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:941.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:941.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692166941168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:941.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:072.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:072.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:47:072.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692167072153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:072.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:47:072.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:072.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:072.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692167072336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:072.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:072.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:072.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692167072628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:072.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:932.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:932.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692169932861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:932.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:933.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:933.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692169933294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:933.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:933.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:49:933.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:49:933.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:933.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692169933724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:933.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:933.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:933.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692169933731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:933.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:933.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:933.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692169933804 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:49:933.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:49:933.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:933.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692169933855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:49:933.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:49:934.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:934.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692169934379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:934.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:934.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:934.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692169934437 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:49:934.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:49:934.465 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:49:985.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:985.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692169985594 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:985.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:986.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:986.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692169986172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40667 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:986.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:986.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:49:987.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:49:987.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:987.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692169987039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:987.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:047.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:047.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692170047083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:047.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:047.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:047.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692170047711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:047.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:048.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:048.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:048.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692170048131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:048.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:048.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692170048137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:048.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:048.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692170048221 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:048.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:048.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692170048324 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:048.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:048.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692170048351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:50:048.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:048.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692170048890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:048.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:048.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:048.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692170048932 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:048.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:048.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:50:100.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:100.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692170100041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:100.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:100.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:100.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170100528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44947 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:100.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:100.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:100.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:100.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:101.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692170100985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:101.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:708.823 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:708.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170708778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:708.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:709.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:709.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170709434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40885 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:709.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:709.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:709.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:709.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:709.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170709820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:709.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:709.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:709.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170709827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:709.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:709.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:709.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170709891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:710.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:710.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:710.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170710267 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:710.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:710.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:710.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692170710309 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:50:710.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:710.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:710.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:50:710.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692170710840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:710.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:711.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:711.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692170710876 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:711.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:757.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:757.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692170757043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:757.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:757.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:757.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692170757237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37133 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:757.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:757.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:757.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:757.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:757.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692170757712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:757.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:977.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:977.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692170977262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:977.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:977.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:977.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692170977744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33059 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:977.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:977.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:978.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:978.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:978.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692170978039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:978.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:109.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:109.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692171109096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:109.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:109.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:109.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692171109605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33081 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:109.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:109.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:109.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:110.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:110.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692171110023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:110.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:117.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:117.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692171117357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:117.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:117.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:117.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692171117871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33895 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:117.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:118.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:118.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:118.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:118.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692171118573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:118.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:118.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:118.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692171118578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:118.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:118.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:118.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692171118640 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:51:118.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:51:118.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:118.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692171118695 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:51:118.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:51:119.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:119.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692171119238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:119.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:119.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:119.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692171119274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:51:119.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:51:119.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:51:171.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:171.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692171171553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:171.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:172.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:172.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692171172002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:172.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:172.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:172.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:172.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:172.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692171172427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:172.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:810.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:811.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692171810914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:811.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:811.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:811.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692171811417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43941 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:811.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:811.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:811.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:811.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:811.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692171811863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:811.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:207.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:207.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692172207573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:207.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:208.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:208.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692172208034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:208.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:208.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:52:208.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:52:208.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:208.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692172208509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:208.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:910.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:910.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692172910199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:910.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:910.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:910.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692172910538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40237 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:910.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:910.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:52:910.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:52:910.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:910.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692172910946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:910.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:922.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:922.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692172922168 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:922.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:922.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:922.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692172922465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38589 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:922.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:922.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:52:922.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:52:922.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:922.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692172922784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:922.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:684.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:684.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692173684068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:684.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:684.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:684.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692173684549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40375 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:684.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:684.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:53:685.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:53:685.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:685.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692173685051 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:685.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:242.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:242.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692174242671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:242.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:243.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:243.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692174243237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41287 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:243.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:243.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:54:243.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:54:243.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:243.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692174243652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:243.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:799.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692176799022 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:799.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:799.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692176799222 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45121 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:799.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:799.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:56:799.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:56:799.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176799694 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:799.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:799.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176799699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:799.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:799.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176799783 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:799.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:799.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176799868 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:799.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:799.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:799.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176799891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:56:799.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:800.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:800.476 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692176800425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:800.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:800.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:800.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692176800486 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:800.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:56:800.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:810.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:811.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176810888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:811.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:56:811.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:56:811.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:811.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46435 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:812.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:811.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:56:812.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:812.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:812.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:812.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:812.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:812.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:812.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811408 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:56:812.272 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:812.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:812.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:812.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:812.317 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692176811927 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:812.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:846.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:846.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692176846650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:846.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:847.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:847.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692176846993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45091 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:847.054 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:847.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:56:847.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:56:847.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:847.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692176847301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:847.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:858.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:858.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692176858029 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:858.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:858.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:858.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692176858485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40053 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:858.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:858.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:56:858.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:56:858.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:858.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692176858946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:859.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:357.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:357.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692177357243 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:357.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:357.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:357.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692177357466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:357.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:357.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:57:357.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:57:357.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692177357966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:358.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:358.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692177357972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:358.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:358.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692177358064 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:57:358.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:57:358.189 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692177358183 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:57:358.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:57:358.229 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692177358210 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:57:358.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:57:358.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692177358755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:358.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:358.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:358.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692177358807 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:57:358.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:57:358.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:57:407.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:408.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692177407960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:408.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:408.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:408.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692177408146 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44203 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:408.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:408.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:57:408.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:57:408.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:408.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692177408530 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:408.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:795.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:795.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692177795133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:795.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:795.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:795.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692177795325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:795.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:795.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:57:795.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:57:795.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:795.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692177795777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:795.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:873.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:873.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692177873392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:873.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:873.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:873.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692177873600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:873.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:873.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:57:873.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:57:873.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:873.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692177873946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:874.001 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:167.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:167.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692178167851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:167.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:168.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:168.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692178168568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38221 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:168.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:168.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:168.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:168.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:169.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178168968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:169.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:169.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:169.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178168975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:169.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:169.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:169.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178169033 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:58:169.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:169.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:169.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:58:169.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178169082 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:58:170.118 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:170.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:170.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178169638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:170.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:170.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:170.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178169673 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:58:170.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:220.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:220.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692178220859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:220.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:221.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:221.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692178221242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:221.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:221.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:221.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:221.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:221.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692178221612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:221.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:389.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:389.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692178389764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:389.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:390.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:390.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692178390425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37993 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:390.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:390.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:390.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:390.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:390.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692178390785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:390.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:202.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:202.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692179202696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:202.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:203.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:203.202 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692179203161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41005 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:203.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:203.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:203.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:203.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:203.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692179203464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:203.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:649.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:650.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692179649889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:650.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:650.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:650.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692179650691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:650.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:651.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:651.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:651.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:651.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692179651160 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:651.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:818.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:818.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692179818092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:818.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:818.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:818.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692179818636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33609 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:818.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:818.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:818.925 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:818.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:819.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692179818964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:819.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:532.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:533.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692180532952 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:533.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:533.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:533.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692180533433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:533.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:533.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:00:534.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:00:534.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:534.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692180534045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:534.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:183.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:183.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692181183649 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:184.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:184.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:184.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692181183834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:184.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:184.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:01:184.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:01:184.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:184.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692181184208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:184.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:828.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:828.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692181828591 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 646 dequeues = 646 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 665 dequeues = 665 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692152841621 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692181184214 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:03:01:828.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:01:828.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:829.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692181828699 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 129 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 147 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 140 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 322, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 646, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:01:829.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:01:836.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:836.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692181836865 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:03:01:836.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:03:01:837.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:837.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692181837078 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1379771104 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.15 five_min_avg = 0.25 fifteen_min_avg = 0.19 ] cpu_share = 1.87832 ] ] 2025-09-01 Mon 02:03:01:837.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:01:837.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:837.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692181837092 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.87832 used_sys_mem = 2854240503 one_min_cpuload = 0.15 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:03:01:837.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:03:01:837.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:837.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692181837134 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 192 in_bytes = 13125 out_tpkts = 208 out_bytes = 11355 exception_packets = 22 exception_packets_allowed = 13 raw_vhost_stats= [ name = vhost0 in_pkts = 192 in_bytes = 13125 out_pkts = 208 out_bytes = 11355 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 177 in_bytes = 10053 out_pkts = 195 out_bytes = 13251 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:01:837.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:01:837.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:837.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692181837169 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:01:837.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:02:841.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:02:841.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:03:02:842.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:02:842.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692182841787 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:02:842.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:02:842.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:02:842.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:02:842.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692182842063 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 129 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 147 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 140 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 322, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 653, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:02:842.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:02:842.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:02:842.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:02:842.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:02:842.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:02:842.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:03:02:842.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:02:842.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692182842563 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 130 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 147 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 140 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 322, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 655, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:02:842.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:02:842.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:02:842.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692182842598 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:02:842.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:02:842.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:02:842.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:02:843.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692182842713 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 130 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 148 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 140 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 323, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 655, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:02:843.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:03:467.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:467.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692183467311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:467.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:467.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:467.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692183467911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:467.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:468.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:03:468.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:03:468.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:468.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692183468325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:468.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:468.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:468.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692183468331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:468.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:468.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:468.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692183468407 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:03:468.467 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:03:468.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:468.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692183468458 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:03:468.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:03:469.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:469.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:03:469.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692183468966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:469.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:469.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:469.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692183469001 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:03:469.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:03:516.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:516.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692183516166 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:516.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:516.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:516.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692183516715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:516.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:517.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:03:517.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:03:517.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:517.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692183517147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:517.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:103.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:103.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184103149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:103.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:103.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:103.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184103330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:103.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:103.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:04:103.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:04:103.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:103.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184103648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:103.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:103.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:103.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184103652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:103.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:103.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:103.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184103710 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:04:103.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:103.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:103.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692184103941 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:04:103.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:104.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:104.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692184103976 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:04:104.044 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:104.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:104.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:04:104.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692184104528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:104.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:104.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:104.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692184104577 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:04:104.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:149.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:149.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692184149659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:149.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:150.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:150.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692184150210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44367 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:150.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:150.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:04:150.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:04:150.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:150.659 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692184150632 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:150.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:498.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:498.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692184498326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:498.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:498.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:498.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692184498792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44085 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:498.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:499.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:04:499.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:04:499.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:499.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692184499339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:499.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:638.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:638.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692184638201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:638.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:638.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:638.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692184638697 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44675 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:638.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:639.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:04:639.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:04:639.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:639.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692184639298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:639.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:639.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:639.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692184639303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:639.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:639.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:639.394 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692184639354 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:04:639.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:639.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:639.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184639454 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:04:639.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:639.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:639.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692184639486 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:04:639.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:640.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:640.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692184640017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:640.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:640.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:640.091 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692184640062 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:04:640.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:04:640.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:04:690.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:690.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692184690316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:690.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:690.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:690.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692184690857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35977 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:690.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:04:691.027 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:04:691.074 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:04:691.130 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:04:691.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692184691120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:04:691.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:068.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:068.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692185068742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:068.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:069.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:069.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692185069229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38257 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:069.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:069.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:069.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:069.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:069.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692185069725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:069.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:216.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:216.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692185216340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:216.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:216.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692185216945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46631 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:217.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:217.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:217.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:217.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692185217224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:217.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:217.274 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692185217230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:217.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:217.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692185217319 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:05:217.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:217.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692185217372 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:05:217.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:217.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692185217913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:217.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:217.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:217.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:05:218.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692185217951 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:05:218.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:267.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:267.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692185267123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:267.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:267.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:267.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692185267302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33827 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:267.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:267.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:267.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:267.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:267.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692185267737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:267.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:672.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:672.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692185672257 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:672.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:672.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:672.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692185672712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:672.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:672.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:672.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:673.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:673.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692185673006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:673.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:266.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:267.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692186266891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:267.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:267.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:267.266 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692186267068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:267.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:267.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:06:267.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:06:267.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:267.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692186267584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:267.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:533.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:533.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692186533486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:533.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:533.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:533.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692186533701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:533.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:533.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:06:534.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:06:534.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:534.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692186534089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:534.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:869.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:870.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692186869876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:870.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:870.213 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:870.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692186870059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:870.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:870.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:06:870.400 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:06:870.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:870.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692186870448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:870.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:599.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:599.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692187599201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:599.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:599.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:599.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692187599768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34303 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:599.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:599.980 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:07:600.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:07:600.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:600.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692187600066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:600.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:391.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:391.838 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692188391729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:391.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:392.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:392.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692188392312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33335 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:392.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:392.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:08:392.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:08:392.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:392.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692188392777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:392.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:134.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:134.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692190134253 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:134.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:134.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:134.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692190134442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43135 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:134.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:134.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:10:134.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:10:134.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:135.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190134859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:135.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:135.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:135.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190134869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:135.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:135.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:135.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190134925 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:10:135.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:135.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:135.704 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190134973 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:10:135.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:135.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:135.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190135506 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:135.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:135.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:135.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190135547 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:10:135.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:135.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:10:182.701 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:182.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692190182670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:182.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:182.883 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:182.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692190182875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:182.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:183.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:10:183.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:10:183.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:183.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692190183135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:183.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:614.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:614.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692190614616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:614.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:614.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:614.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692190614799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:614.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:615.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:10:615.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:10:615.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:615.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615207 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:615.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:615.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:615.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:615.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:615.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:616.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615272 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:10:616.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:616.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:10:616.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:616.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615375 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:10:616.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:616.184 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:616.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615395 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:10:616.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:616.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:616.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:616.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:616.249 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:616.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692190615980 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:10:616.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:10:664.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:664.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692190664132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:664.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:664.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:664.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692190664335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33467 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:664.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:664.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:10:664.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:10:664.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:664.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692190664768 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:664.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:209.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:209.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692191209320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:209.452 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:209.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:11:209.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:209.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692191209554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:209.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:209.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:11:209.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:210.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692191209977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:210.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:524.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:524.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692191524214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:524.335 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:524.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:524.890 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692191524843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35171 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:524.897 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:525.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:11:525.100 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:11:525.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:525.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692191525143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:525.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:525.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:525.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692191525149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:525.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:525.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:525.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692191525206 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:11:525.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:525.484 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:525.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692191525475 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:11:525.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:525.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:525.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692191525512 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:11:525.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:526.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:526.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692191526555 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:526.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:526.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:526.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692191526604 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:11:526.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:526.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:11:572.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:572.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692191572741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:572.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:573.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:573.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692191573209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39679 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:573.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:573.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:11:573.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:11:573.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:573.794 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692191573757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:573.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:582.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:582.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692191582894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:582.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:583.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:583.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692191583516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:583.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:583.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:11:583.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:11:583.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:583.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692191583895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:583.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:460.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:461.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692192460897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:461.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:461.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:461.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692192461123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:461.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:461.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:12:461.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:12:461.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:461.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692192461602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:461.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:461.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:461.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692192461610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:461.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:461.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:461.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692192461714 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:12:461.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:12:461.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:461.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692192461774 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:12:461.812 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:12:462.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:462.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692192462601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:462.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:12:462.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:462.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:462.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692192462646 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:12:462.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:12:511.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:511.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692192511726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:511.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:511.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:511.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692192511910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36763 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:511.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:512.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:12:512.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:12:512.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:512.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692192512345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:512.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:545.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:546.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692192545906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:546.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:546.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:546.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692192546080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37491 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:546.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:546.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:12:546.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:12:546.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:546.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692192546475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:546.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:118.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:118.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692193118167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:118.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:118.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:118.415 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692193118381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43309 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:118.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:118.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:13:118.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:13:118.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:118.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692193118693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:118.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:447.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:45996 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:13:456.517 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:46000 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:13:457.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:457.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692193457421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:457.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:457.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:457.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692193457590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37023 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:457.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:457.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:13:457.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:13:457.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:457.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692193457915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:457.941 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:636.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:636.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692193636032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:636.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:636.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:636.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692193636661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:636.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:636.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:13:637.017 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:13:637.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:637.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692193637055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:637.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:652.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:652.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692194652591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:652.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:653.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:653.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692194653092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37631 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:653.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:653.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:14:653.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:14:653.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:653.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692194653611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:653.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:276.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:276.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692195276068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:276.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:276.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:276.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692195276261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44193 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:276.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:276.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:15:276.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:15:276.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:276.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692195276654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:276.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:934.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:935.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692196934846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:935.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:935.128 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:935.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692196935027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40903 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:935.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:935.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:16:935.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:16:935.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:935.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692196935332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:935.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:935.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:935.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692196935339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:935.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:935.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:935.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692196935431 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:16:935.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:16:935.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:935.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692196935487 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:16:935.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:16:936.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:936.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692196936100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:936.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:936.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:936.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692196936147 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:16:936.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:16:936.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:16:984.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:984.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692196984295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:984.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:984.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:984.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692196984477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:984.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:984.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:16:984.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:16:984.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:984.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692196984927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:984.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:237.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:237.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692197237221 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:237.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:237.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:237.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692197237670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37429 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:237.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:237.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:17:238.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:17:238.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:238.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692197238126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:238.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:238.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:238.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692197238136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:238.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:238.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:238.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692197238244 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:17:238.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:17:238.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:238.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692197238344 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:17:238.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:17:238.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:238.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692197238361 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:17:238.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:17:238.957 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:238.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692197238915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:238.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:238.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:239.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692197238956 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:17:239.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:17:238.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:17:289.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:289.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692197289143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:289.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:289.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:289.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692197289743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34633 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:289.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:289.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:17:290.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:17:290.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:290.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692197290150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:290.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:056.171 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:056.489 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692198056136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:056.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:056.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:056.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692198056323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42127 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:056.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:056.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:056.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:056.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:056.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198056728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:056.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:253.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:253.652 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198253347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:253.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:253.710 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:253.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:253.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:253.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198253545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45397 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:253.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:253.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:254.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198253925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:254.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:865.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:865.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692198865732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:865.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:866.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:866.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198866430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:866.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:866.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:866.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:866.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:866.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692198866831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:866.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:866.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:867.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692198866838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:867.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:867.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:867.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692198866900 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:867.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:867.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:867.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692198867101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:867.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:867.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:867.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692198867130 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:18:867.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:867.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:867.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198867673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:867.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:867.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:867.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198867728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:867.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:867.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:18:876.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:876.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692198876744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:876.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:877.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:877.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198877269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36283 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:877.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:877.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:877.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:877.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:877.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198877658 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:877.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:877.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:877.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198877663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:877.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:877.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:877.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198877726 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:877.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:877.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:877.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198877769 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:18:877.820 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:878.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:878.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198878246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:878.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:878.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:878.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692198878279 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:878.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:878.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:18:920.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:921.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692198920939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:921.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:921.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:921.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692198921126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42969 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:921.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:921.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:921.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:921.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:921.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692198921546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:921.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:923.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:923.432 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692198923367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:923.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:923.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:923.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692198923656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36567 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:923.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:923.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:923.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:923.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:923.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692198923918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:923.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:875.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:876.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692199875790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:876.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:876.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:876.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692199875953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:876.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:876.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:19:876.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:19:876.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:876.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692199876368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:876.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:950.105 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:950.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692199950076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:950.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:950.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:19:950.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:950.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692199950275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46827 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:950.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:950.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:19:950.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:950.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692199950737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:950.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:982.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:982.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692199982837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:982.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:983.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:983.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692199983259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35517 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:983.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:983.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:19:983.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:19:983.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:983.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692199983752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:983.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:234.123 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:234.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:20:234.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692200234078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:234.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:234.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:20:234.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:234.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692200234275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39433 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:234.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:234.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:234.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692200234604 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:234.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:966.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:966.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692201966800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:967.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:967.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:967.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692201967007 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39343 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:967.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:967.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:21:967.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:21:967.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:967.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692201967301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:967.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:000.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:000.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692202000521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:000.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:001.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:001.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692202001052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42633 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:001.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:001.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:22:001.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:22:001.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:001.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692202001445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:001.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:655.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:656.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692203655932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:656.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:656.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:656.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692203656133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41035 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:656.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:656.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:23:656.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:23:656.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:656.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692203656585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:656.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:656.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:656.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692203656590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:656.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:656.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:656.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203656728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:23:656.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:656.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:656.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203656785 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:23:656.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:657.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:657.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692203657337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:657.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:657.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:657.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:23:657.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692203657377 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:23:657.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:703.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:703.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692203703572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:703.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:704.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:704.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692203704153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42803 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:704.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:704.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:23:704.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:23:704.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:704.616 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692203704580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:704.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:942.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:942.992 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203942746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:943.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:943.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:943.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:23:943.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:23:943.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203942928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:943.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:943.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:943.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203943245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:943.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:943.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:943.577 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203943249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:943.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:943.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:943.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203943344 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:23:943.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:943.718 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:943.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692203943705 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:23:943.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:943.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:943.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692203943725 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:23:943.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:944.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:944.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203944534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:944.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:944.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:944.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:23:944.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203944599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:23:944.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:23:995.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:995.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692203995753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:995.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:996.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:996.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692203996187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:996.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:996.589 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:23:996.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:23:996.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:996.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692203996791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:996.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:631.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:631.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692204631759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:631.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:632.419 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:632.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692204632395 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:632.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:632.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:24:632.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:24:632.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:632.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692204632772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:632.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:905.939 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:906.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692204905907 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:906.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:906.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:906.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692204906482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:906.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:906.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:24:906.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:24:906.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:906.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692204906919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:906.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:603.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:603.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692205603467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:603.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:603.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:604.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692205603989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:604.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:604.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:25:604.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:25:604.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:604.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692205604396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:604.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:604.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:604.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692205604402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:604.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:604.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:604.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692205604461 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:25:604.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:25:604.524 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:604.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692205604507 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:25:604.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:25:605.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:605.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692205605075 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:605.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:605.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:605.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692205605132 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:25:605.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:25:605.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:25:650.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:650.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692205650295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:650.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:650.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:650.541 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692205650517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:650.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:25:650.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:25:650.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:25:651.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:651.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692205650973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:25:651.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:285.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:285.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692206285653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:285.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:285.849 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:285.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692206285841 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:285.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:286.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:286.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:286.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:286.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692206286298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:286.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:286.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:286.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692206286304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:286.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:286.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:286.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692206286369 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:286.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:286.518 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:286.540 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692206286510 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:286.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:286.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:286.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692206286533 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:26:286.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:287.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:287.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692206287165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:287.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:287.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:287.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:26:287.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692206287208 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:287.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:336.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:336.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692206336307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:336.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:336.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:336.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692206336836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:336.874 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:337.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:337.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:337.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:337.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692206337314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:337.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:578.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:578.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692206578088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:578.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:578.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:578.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692206578703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:578.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:579.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:579.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:579.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:579.175 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692206579143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:579.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:648.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:649.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692206648926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:649.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:649.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:649.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692206649354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43241 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:649.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:649.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:649.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:649.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:649.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692206649792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:649.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:743.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:743.141 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692206743037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:743.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:743.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:743.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692206743236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:743.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:743.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:743.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:743.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:743.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692206743671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:743.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:291.539 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:291.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692207291505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:291.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:292.003 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:292.032 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692207291990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:292.040 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:292.239 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:27:292.342 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:27:292.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:292.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692207292378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:292.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:451.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:451.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692208451264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:451.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:451.728 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:451.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692208451718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42779 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:451.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:452.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:28:452.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:28:452.301 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:452.320 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692208452287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:452.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:416.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:416.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692209416561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:416.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:417.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:29:417.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:29:417.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:417.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692209416747 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:417.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:417.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:417.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692209417362 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:417.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:825.986 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:826.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692210825945 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:826.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:826.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:826.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210826517 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:826.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:826.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:30:827.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:30:827.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:827.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692210827096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:827.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:827.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:827.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692210827103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:827.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:827.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:827.240 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692210827205 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:30:827.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:827.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:827.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692210827260 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:30:827.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:827.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:827.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210827901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:827.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:827.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:827.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210827958 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:30:828.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:827.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:30:847.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:847.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210847741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:847.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:847.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:847.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210847888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35027 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:847.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:848.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:30:848.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:30:848.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:848.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692210848283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:848.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:848.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:848.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692210848288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:848.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:848.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:848.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692210848339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:30:848.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:848.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:848.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692210848564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:30:848.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:848.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:848.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692210848583 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:30:848.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:849.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:849.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692210849106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:849.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:849.169 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:849.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:30:849.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692210849143 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:30:849.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:30:873.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:873.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210873049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:873.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:873.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:873.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692210873600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:873.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:873.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:30:873.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:30:873.924 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:873.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692210873916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:873.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:902.293 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:902.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210902271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:902.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:902.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:902.461 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692210902441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40737 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:902.468 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:902.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:30:902.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:30:902.856 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:902.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692210902847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:902.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:803.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:803.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692211803028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:803.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:803.678 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:803.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692211803646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35663 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:803.739 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:803.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:31:803.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:31:804.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:804.160 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692211804035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:804.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:829.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:829.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692211828866 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 964 dequeues = 964 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 986 dequeues = 986 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692182842670 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692211804046 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:03:31:829.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:31:829.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:829.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692211828984 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 194 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 219 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 731 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 209 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 481, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 964, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:31:829.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:31:837.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:837.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692211837333 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:03:31:837.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:03:31:837.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:837.604 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692211837377 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 294 in_bytes = 30282 out_tpkts = 316 out_bytes = 17587 exception_packets = 25 exception_packets_dropped = 10 exception_packets_allowed = 15 raw_vhost_stats= [ name = vhost0 in_pkts = 294 in_bytes = 30282 out_pkts = 316 out_bytes = 17587 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 6.98226e-15 total_out_bandwidth_utilization = 1.67238e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1288, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3085, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 270 in_bytes = 15655 out_pkts = 297 out_bytes = 30408 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:31:837.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:31:837.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:837.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692211837405 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:31:837.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:31:864.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:864.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692211864043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:864.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:864.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:864.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692211864259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:864.302 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:864.548 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:31:864.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:31:864.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:864.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692211864656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:864.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:412.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:412.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692212412467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:412.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:412.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:413.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692212412989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:413.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:413.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:32:413.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:32:413.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:413.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692212413657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:413.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:413.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:413.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692212413664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:413.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:413.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:413.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692212413724 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:32:413.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:32:413.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:413.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692212413772 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:32:413.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:32:414.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:414.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692212414294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:414.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:414.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:414.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692212414341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:32:414.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:32:414.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:32:464.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:464.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212464679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:464.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:464.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:464.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212464866 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:464.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:465.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:32:465.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:32:465.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:465.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692212465348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:32:465.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:842.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:32:843.104 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:03:32:843.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:32:843.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212842912 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:32:843.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:32:843.328 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:32:843.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:32:843.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:32:843.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212843224 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 197 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 222 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 742 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 212 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 488, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 983, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:32:843.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:32:843.778 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:32:843.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:32:843.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:32:843.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Idle 2025-09-01 Mon 02:03:32:844.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:844.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212843869 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 198 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 222 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 742 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 212 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 488, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 985, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:32:844.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:32:844.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:844.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212843909 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:32:844.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:32:844.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:32:844.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:32:844.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692212844021 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 198 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 223 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 742 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 212 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 985, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:32:844.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:33:233.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:233.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692213233510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:233.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:234.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:234.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692213234177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36271 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:234.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:234.436 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:234.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:234.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:234.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692213234553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:234.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:234.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:234.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692213234557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:234.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:234.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:234.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692213234612 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:234.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:234.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:234.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692213234692 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:234.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:234.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:234.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692213234709 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:33:234.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:235.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:235.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692213235281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:235.333 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:235.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:235.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692213235324 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:235.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:235.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:33:287.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:287.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692213287468 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:287.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:287.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:287.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692213287879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:287.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:288.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:288.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:288.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:288.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692213288483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:288.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:491.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:491.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692213491513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:491.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:492.196 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:492.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692213492147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41917 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:492.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:492.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:492.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:492.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:492.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692213492584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:492.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:718.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:718.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692213718803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:718.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:719.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:719.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692213719277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:719.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:719.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:719.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:719.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:719.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692213719683 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:719.726 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:856.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:856.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692213856183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:856.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:856.482 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:856.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692213856363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:856.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:856.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:856.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:856.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:856.807 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692213856774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:856.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:251.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:251.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692214251616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:251.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:252.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:252.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692214252179 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:252.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:252.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:34:252.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:34:252.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:252.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692214252720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:252.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:382.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:382.836 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692215382730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:382.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:383.216 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:383.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692215383205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38585 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:383.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:383.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:35:383.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:35:383.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:383.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692215383826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:383.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:36:358.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:36:358.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692216358893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:36:359.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:36:359.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:36:359.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692216359524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33933 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:36:359.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:36:359.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:36:360.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:36:360.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:36:360.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692216360216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:36:360.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:673.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:673.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692217672980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:673.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:673.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:673.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692217673487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46287 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:673.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:673.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:37:673.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:37:673.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:673.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692217673903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:673.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:673.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:673.998 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692217673912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:674.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:674.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:674.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692217673993 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:37:674.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:674.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:674.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692217674078 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:37:674.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:674.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:674.164 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692217674096 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:37:674.170 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:674.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:674.744 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692217674681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:674.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:674.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:674.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692217674728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:37:674.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:674.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:37:720.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:721.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692217720904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:721.019 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:721.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:721.733 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692217721659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:721.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:721.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:37:722.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:37:722.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:722.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692217722595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:722.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:787.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:787.911 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692217787807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:787.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:788.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:788.287 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692217788233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38107 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:788.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:788.454 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:37:788.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:37:789.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:37:789.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:789.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217788571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:789.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:789.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:789.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217788586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:789.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:789.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:789.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217788649 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:37:789.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:789.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:789.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217788706 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:37:789.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:789.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:789.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217789295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:789.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:789.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:789.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217789343 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:37:789.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:37:835.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:835.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692217835493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:835.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:835.682 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:835.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217835674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37647 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:835.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:836.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:37:836.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:37:836.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:836.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692217836161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:836.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:772.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:772.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692218772286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:772.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:773.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:773.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692218773365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38877 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:773.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:773.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:38:773.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:38:773.833 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:773.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692218773788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:773.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:793.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:793.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692218793659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:793.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:794.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:794.362 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692218794327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33731 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:794.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:794.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:38:794.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:38:794.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:794.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692218794641 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:794.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:416.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:416.250 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692219415989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:416.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:416.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:416.303 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692219416179 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:416.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:416.449 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:39:416.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:39:416.562 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:416.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692219416554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:416.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:416.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:416.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692219416559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:416.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:416.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:416.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692219416632 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:39:416.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:39:416.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:416.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692219416696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:39:416.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:39:417.259 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:417.283 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692219417212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:417.291 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:417.298 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:417.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692219417251 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:39:417.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:39:417.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:39:468.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:468.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692219468388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:468.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:468.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:468.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692219468845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33945 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:468.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:469.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:39:469.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:39:469.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:469.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692219469242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:469.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:428.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:428.497 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692220428376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:428.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:428.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:428.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692220428855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38799 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:428.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:429.204 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:429.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:429.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:429.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692220429325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:429.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:429.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:429.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692220429332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:429.402 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:429.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:429.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692220429434 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:429.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:429.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:430.008 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692220429978 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:430.018 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:430.026 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:430.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692220430007 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:40:430.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:430.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:430.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692220430568 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:430.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:430.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:430.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692220430595 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:430.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:430.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:40:479.847 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:479.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692220479814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:479.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:480.334 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:480.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692220480325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33489 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:480.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:480.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:480.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:480.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:480.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692220480763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:480.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:540.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:540.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692220540426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:540.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:541.069 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:541.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692220541042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:541.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:541.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:541.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:541.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:541.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692220541380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:541.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:702.827 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:702.905 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692220702790 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:702.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:703.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:703.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692220703270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:703.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:703.496 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:703.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:703.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:703.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692220703622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:703.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:807.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:808.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692220807938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:808.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:808.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:808.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692220808491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44095 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:808.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:808.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:809.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:809.192 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:809.227 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692220809179 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:809.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:426.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:426.085 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692221425918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:426.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:426.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:426.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692221426230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36229 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:426.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:426.478 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:41:426.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:41:426.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:426.590 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692221426557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:426.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:395.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:395.578 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692222395496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:395.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:395.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:395.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692222395685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:395.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:42:395.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:42:395.930 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:42:395.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:42:395.997 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692222395962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:42:396.006 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:568.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:568.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692223568702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:568.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:569.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:569.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692223569311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43133 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:569.393 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:43:569.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:43:569.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:43:569.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:569.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692223569741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:43:569.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:627.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:627.871 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692224627765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:627.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:628.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:628.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692224628406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:628.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:628.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:44:628.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:44:628.725 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:628.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692224628716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:628.775 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:628.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:628.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692224628723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:628.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:628.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:628.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692224628800 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:44:628.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:44:628.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:628.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692224628899 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:44:628.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:44:628.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:628.977 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692224628923 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:44:628.990 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:44:629.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:44:629.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:629.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692224629491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:629.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:629.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:629.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692224629654 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:44:629.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:44:675.891 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:675.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692224675855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:675.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:676.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:676.533 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692224676451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40045 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:676.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:44:676.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:44:677.225 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:44:677.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:677.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692224677253 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:44:677.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:057.364 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:057.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692225057324 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:057.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:057.662 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:057.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692225057515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33685 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:057.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:057.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:45:057.915 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:45:057.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:058.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225057967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:058.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:058.761 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:45:058.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:058.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225058003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:058.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:058.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:058.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225058066 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:45:058.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:45:058.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:058.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225058128 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:45:059.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:45:059.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:059.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225058700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:059.038 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:059.047 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:059.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225058737 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:45:059.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:45:107.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:107.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225107852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:107.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:108.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:108.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692225108022 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:108.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:108.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:45:108.481 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:45:108.529 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:108.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692225108519 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:108.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:748.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:748.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692225748455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:748.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:749.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:749.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692225749095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:749.190 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:45:749.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:45:749.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:45:749.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:45:749.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692225749674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:45:749.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:134.745 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:134.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692226134686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:134.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:135.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:135.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692226135360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38965 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:135.443 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:135.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:46:136.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:46:136.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:136.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226136095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:136.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:644.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:644.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:46:644.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:46:644.615 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226644094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:644.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:645.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226644276 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:645.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:645.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226644656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:645.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:645.644 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226644660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:645.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:645.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226644741 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:46:645.708 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:46:645.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226644795 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:46:645.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:46:645.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226645357 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:645.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:645.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:645.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692226645422 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:46:645.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:46:645.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:46:696.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:696.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692226696529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:696.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:697.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:697.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692226697156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:697.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:46:697.595 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:46:697.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:46:697.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:46:697.747 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692226697711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:46:697.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:421.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:422.021 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692227421896 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:422.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:422.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:422.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692227422546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42589 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:422.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:422.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:422.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:422.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692227422966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:423.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:423.144 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692227422976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:423.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:423.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.201 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692227423039 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:47:423.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:423.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692227423288 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:47:423.348 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:423.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692227423308 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:47:423.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:423.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692227423900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:423.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:423.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:423.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692227423954 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:47:423.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:47:424.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:47:469.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:469.219 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692227469107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:469.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:469.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:469.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692227469572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33755 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:469.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:469.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:469.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:469.927 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:469.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692227469916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:469.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:711.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:711.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692227711828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:711.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:712.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:712.281 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692227712254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40905 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:712.288 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:712.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:712.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:712.750 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:712.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692227712737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:712.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:723.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:723.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692227723796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:723.861 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:724.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:724.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692227724065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:724.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:47:724.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:47:724.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:47:724.376 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:47:724.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692227724348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:47:724.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:170.306 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:170.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692228170274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:170.632 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:170.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:170.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692228170451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36307 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:170.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:170.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:48:170.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:48:170.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:170.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692228170854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:170.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:379.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:379.341 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692228379054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:379.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:379.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:379.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:48:379.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:48:379.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692228379244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43013 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:379.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:48:379.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:48:379.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692228379564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:48:379.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:650.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:650.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692229650548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:650.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:650.876 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:650.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692229650732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:650.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:49:651.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:49:651.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:49:651.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:49:651.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692229651210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:49:651.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:359.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:359.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692230359749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:359.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:360.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:360.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692230360202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:360.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:50:360.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:50:360.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:50:360.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:50:360.614 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692230360585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:50:360.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:492.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:492.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692231492869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:493.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:493.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:493.463 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692231493434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36987 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:493.477 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:493.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:51:493.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:51:493.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231493971 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:494.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:494.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231493978 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:494.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:494.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692231494074 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:51:494.138 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:494.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.206 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692231494173 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:51:494.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:494.226 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692231494196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:51:494.248 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:494.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231494753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:494.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:494.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:494.817 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231494792 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:51:494.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:494.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:51:544.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:544.995 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692231544894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:545.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:545.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:545.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692231545373 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:545.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:545.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:51:545.813 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:51:545.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:545.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692231545893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:545.951 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:915.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:915.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692231915008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:915.127 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:915.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:915.314 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692231915291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36823 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:915.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:915.637 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:51:915.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:51:915.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:915.845 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231915820 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:915.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:915.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:915.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231915825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:915.886 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:915.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:915.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231915879 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:51:915.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:915.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:915.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692231915928 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:51:915.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:916.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:916.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692231916445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:916.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:916.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:916.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692231916483 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:51:916.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:51:916.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:51:964.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:964.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692231964847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:964.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:965.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:965.473 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692231965396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34525 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:965.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:965.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:51:965.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:51:965.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:965.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692231965731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:965.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:527.042 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:527.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692232527003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:527.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:527.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:527.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692232527466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:527.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:527.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:52:527.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:52:527.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:527.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692232527938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:527.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:937.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:937.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692232937857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:937.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:938.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:938.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692232938305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:938.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:52:938.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:52:938.656 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:52:938.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:52:938.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692232938702 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:52:938.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:611.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:611.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692233611330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:611.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:611.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:611.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692233611513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39921 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:611.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:611.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:53:611.914 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:53:611.985 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:612.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692233611976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:612.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:612.783 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:612.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692233611981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:612.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:612.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:612.828 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692233612043 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:53:612.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:53:612.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:612.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692233612100 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:53:612.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:53:612.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:612.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692233612652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:612.901 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:612.907 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:612.922 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692233612697 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:53:612.931 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:53:612.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:53:658.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:658.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692233658879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:659.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:659.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:659.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692233659461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41129 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:659.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:53:659.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:53:659.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:53:659.858 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:53:659.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692233659849 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:53:659.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:320.786 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:320.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692234320735 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:320.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:320.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:320.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692234320947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40545 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:321.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:321.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:321.292 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:321.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:321.367 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692234321332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:321.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:321.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:321.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692234321339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:321.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:321.426 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:321.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692234321420 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:54:321.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:321.603 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:321.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692234321594 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:54:321.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:321.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:321.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692234321621 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:54:321.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:322.326 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:322.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692234322291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:322.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:322.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:322.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692234322375 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:54:322.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:322.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:54:369.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:369.661 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692234369513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:369.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:370.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:370.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692234370086 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:370.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:370.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:370.521 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:370.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:370.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692234370570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:370.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:544.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:544.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692234544070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:544.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:544.549 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:544.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692234544542 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46435 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:544.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:544.863 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:544.953 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:545.020 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:545.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692234544983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:545.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:605.051 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:605.121 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692234605015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:605.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:605.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:605.727 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692234605650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37317 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:605.736 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:605.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:605.979 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:606.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:606.066 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692234606015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:606.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:792.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:793.049 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692234792925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:793.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:793.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:793.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692234793451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:793.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:793.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:793.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:793.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:793.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692234793836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:793.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:423.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:423.784 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692235423679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:423.795 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:424.136 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:424.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692235424129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42001 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:424.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:424.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:55:424.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:424.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:424.657 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692235424589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:424.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:622.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:622.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692236622202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:622.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:622.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:622.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692236622863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:622.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:56:623.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:56:623.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:56:623.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:623.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692236623523 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:56:623.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:548.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:549.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692237548739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:549.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:549.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:549.086 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692237548918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41373 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:549.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:549.260 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:549.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:549.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:549.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692237549361 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:549.406 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:502.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:502.147 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692238502038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:502.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:502.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:502.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692238502612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:502.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:502.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:58:502.898 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:58:502.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:503.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238502932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:503.173 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:503.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:503.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238502940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:503.207 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:503.214 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:503.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238503037 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:58:503.262 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:503.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:503.304 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238503095 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:58:503.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:503.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:503.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692238503596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:503.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:503.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:503.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:58:503.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692238503621 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:58:503.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:505.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:505.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238505054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:505.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:505.586 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:505.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692238505567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:505.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:505.808 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:58:505.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:58:505.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:505.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692238505866 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:505.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:505.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:505.923 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692238505870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:505.929 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:505.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:505.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692238505917 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:58:506.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:506.062 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:506.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692238505998 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:58:506.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:506.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:506.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692238506018 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:58:506.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:506.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:506.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692238506457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:506.535 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:506.546 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:506.575 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692238506492 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:58:506.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:58:506.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:58:554.909 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:555.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:58:556.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238554785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:556.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:556.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:556.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238555380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:556.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:556.585 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:58:556.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:556.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692238556612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:556.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:556.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:556.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692238556801 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:556.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:557.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:557.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692238557131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39555 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:557.168 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:58:557.267 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:58:557.309 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:58:557.339 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:58:557.352 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692238557327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:58:557.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:59:466.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:59:466.580 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692239466455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:59:466.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:59:466.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:59:466.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692239466654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:59:466.719 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:59:466.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:59:466.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:59:467.000 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:59:467.025 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692239466989 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:59:467.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:59:546.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:59:546.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692239546791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:59:546.899 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:59:547.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:59:547.237 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692239547210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39199 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:59:547.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:59:547.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:59:547.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:59:547.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:59:547.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692239547700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:59:547.768 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:439.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:439.822 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692240439722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:439.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:439.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:439.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692240439926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39063 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:439.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:440.252 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:00:440.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:00:440.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:440.392 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692240440356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:440.401 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:440.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:440.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692240440363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:440.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:440.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:440.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692240440444 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:00:440.487 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:00:440.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:440.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692240440495 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:00:440.523 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:00:441.050 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:441.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692240441009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:441.077 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:441.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:441.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:00:441.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692240441075 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:00:441.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:00:489.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:489.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692240489201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:489.305 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:489.412 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:489.430 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692240489404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:489.437 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:489.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:00:489.685 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:00:489.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:489.785 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692240489753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:489.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:401.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:401.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692241401557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:401.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:402.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:402.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692241402106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:402.157 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:402.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:402.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:402.680 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:402.705 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692241402667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:402.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:402.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:402.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692241402677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:402.760 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:402.767 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:402.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692241402745 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:402.805 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:402.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:402.913 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241402857 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:402.926 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:402.938 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:402.960 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241402882 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:01:402.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:403.510 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:403.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692241403501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:403.534 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:403.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:403.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692241403549 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:403.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:403.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:01:451.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:451.955 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692241451848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:451.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:452.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:452.351 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692241452319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:452.358 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:452.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:452.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:452.771 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:452.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692241452751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:452.799 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:461.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:461.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692241461863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:461.963 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:462.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:462.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692241462192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:462.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:462.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:462.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:462.460 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:462.480 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692241462448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:462.488 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:483.149 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:483.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692241483107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:483.221 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:483.584 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:483.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692241483576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:483.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:483.732 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:483.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:483.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:483.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692241483834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:483.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:545.881 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:545.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692241545838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:545.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:546.504 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:546.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692241546469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:546.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:546.880 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:547.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:547.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:547.366 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692241547322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:547.374 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:829.312 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:829.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692241829141 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 1289 dequeues = 1289 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 1314 dequeues = 1314 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692212843981 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692241547332 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:04:01:829.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:01:829.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:829.566 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692241829282 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 259 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 292 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 983 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 281 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 644, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1289, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:01:829.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:01:837.796 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:837.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241837765 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:04:01:837.853 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:04:01:837.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:837.900 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241837822 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 1371374304 used = 32687 free = 0 buffers = 0 cached = 1440759480 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.055 five_min_avg = 0.2 fifteen_min_avg = 0.175 ] cpu_share = 1.73333 ] ] 2025-09-01 Mon 02:04:01:837.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:01:837.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:837.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241837830 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.73333 used_sys_mem = 2854240503 one_min_cpuload = 0.055 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:04:01:837.950 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:04:01:837.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:838.110 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241837857 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 386 in_bytes = 36450 out_tpkts = 418 out_bytes = 23171 exception_packets = 28 exception_packets_allowed = 18 raw_vhost_stats= [ name = vhost0 in_pkts = 386 in_bytes = 36450 out_pkts = 418 out_bytes = 23171 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 357 in_bytes = 20609 out_pkts = 389 out_bytes = 36576 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:01:838.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:01:838.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:838.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692241837882 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:01:838.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:02:433.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:434.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692242433914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:434.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:434.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:434.608 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692242434524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45289 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:434.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:434.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:02:434.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:02:434.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:434.968 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692242434938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:02:434.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:02:844.133 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:02:844.448 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:04:02:844.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:02:844.677 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:02:844.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:02:844.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692242844233 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:02:844.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:02:844.962 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:02:845.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692242844583 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 259 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 293 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 986 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 282 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 646, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1299, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:02:845.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:02:845.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:02:845.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:02:845.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:02:845.331 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Idle 2025-09-01 Mon 02:04:02:845.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:845.558 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692242845234 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 293 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 986 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 282 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 646, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1301, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:02:845.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:02:845.592 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:845.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692242845274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:02:845.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:02:845.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:02:845.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:02:845.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692242845367 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 294 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 986 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 282 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 647, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1301, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:02:845.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:03:388.606 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:388.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692243388566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:388.686 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:388.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:388.774 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692243388750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:388.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:388.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:03:389.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:03:389.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:03:389.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692243389125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:03:389.174 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:04:559.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:04:559.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692244559095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:04:559.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:04:559.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:04:559.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692244559609 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:04:559.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:04:559.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:04:560.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:04:560.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:04:560.087 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692244560041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:04:560.096 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:219.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:219.551 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692245219444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:219.567 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:220.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:220.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692245220069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:220.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:220.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:05:220.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:05:220.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:220.547 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692245220499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:220.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:220.564 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:220.573 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692245220508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:220.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:220.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:220.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692245220635 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:05:220.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:220.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:220.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692245220698 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:05:220.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:221.318 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:221.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:05:221.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692245221263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:221.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:221.648 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:221.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692245221305 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:05:221.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:269.639 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:269.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692245269501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:269.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:269.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:269.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692245269932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:269.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:270.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:05:270.384 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:05:270.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:270.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692245270422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:270.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:516.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:516.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692245516049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:516.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:516.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:516.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692245516645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:516.707 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:516.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:05:516.888 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:05:516.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:516.958 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692245516939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:516.964 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:516.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:516.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692245516944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:516.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:517.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:517.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692245517003 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:05:517.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:517.099 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:517.140 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692245517086 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:05:517.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:517.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:517.199 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692245517106 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:05:517.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:517.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:517.776 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692245517704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:517.789 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:517.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:517.803 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:05:517.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692245517762 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:05:517.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:05:566.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:567.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692245566964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:567.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:567.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:567.597 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692245567536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38987 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:567.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:05:567.851 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:05:567.967 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:05:568.016 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:05:568.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692245568005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:05:568.053 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:170.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:170.687 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692246170591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:170.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:171.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:171.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692246171167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40691 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:171.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:171.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:06:171.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:06:171.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:171.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692246171547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:171.583 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:477.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:477.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692246477183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:477.296 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:477.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:477.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692246477860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40309 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:477.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:06:478.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:06:478.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:06:478.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:06:478.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692246478395 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:06:478.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:493.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:493.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692247493316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:493.444 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:494.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:494.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692247494085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45891 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:494.203 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:494.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:07:494.373 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:07:494.435 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:494.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692247494423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:495.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:495.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:495.052 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692247494432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:495.061 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:495.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:495.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692247494494 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:07:495.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:07:495.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:495.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692247494543 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:07:495.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:07:495.211 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:495.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692247495094 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:495.245 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:495.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:495.284 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692247495158 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:07:495.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:07:495.186 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:07:540.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:540.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692247540315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:540.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:540.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:540.974 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692247540920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34867 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:540.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:07:541.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:07:541.570 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:07:541.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:07:541.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692247541618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:07:541.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:187.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:187.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248187674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:187.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:188.337 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:188.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692248188306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40601 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:188.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:188.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:188.647 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:188.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:188.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692248188682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:188.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:296.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:296.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692248296559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:296.697 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:297.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:297.151 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692248297107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:297.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:297.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:297.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:297.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:297.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248297656 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:297.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:340.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:340.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692248340237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:340.344 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:340.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:340.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692248340807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36623 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:340.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:341.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:341.379 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:341.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:341.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248341417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:341.462 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:341.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:341.479 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248341425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:341.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:341.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:341.587 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248341541 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:341.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:341.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:341.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248341651 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:341.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:341.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:341.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248341669 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:08:341.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:342.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:342.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692248342204 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:342.271 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:342.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:342.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:08:342.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692248342247 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:08:342.409 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:08:389.414 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:389.493 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692248389375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:389.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:389.628 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:389.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692248389616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41987 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:389.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:389.870 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:389.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:390.002 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:390.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692248389992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:390.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:486.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:486.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692248486812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:486.935 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:487.471 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:487.520 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692248487442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:487.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:487.770 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:487.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:487.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:487.895 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692248487853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:487.902 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:398.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:398.316 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692249398192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:398.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:398.818 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:398.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692249398793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 32907 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:398.885 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:399.067 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:09:399.135 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:09:399.222 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:399.236 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692249399170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:399.244 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:540.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:540.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692250540032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:540.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:540.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:540.740 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692250540657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46499 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:540.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:10:541.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:10:541.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:10:541.119 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:10:541.142 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692250541102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:10:541.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:217.338 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:217.434 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692251217296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:217.457 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:217.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:217.781 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692251217753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35047 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:217.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:217.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:11:218.071 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:11:218.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:218.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692251218105 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:218.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:113.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:114.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692252113804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:114.162 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:114.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:114.193 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692252113987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36947 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:114.205 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:114.325 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:12:114.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:12:114.451 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:114.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692252114441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:114.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:114.617 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:114.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692252114446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:114.633 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:114.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:114.663 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692252114510 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:12:114.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:115.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:115.455 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252114735 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:12:115.474 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:115.483 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:115.500 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252114766 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:12:115.507 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:115.514 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:115.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252115346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:115.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:115.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:115.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252115377 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:12:115.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:115.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:12:167.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:167.841 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692252167746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:167.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:167.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:167.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692252167929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:167.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:168.215 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:12:168.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:12:168.324 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:168.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692252168314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:168.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:400.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:401.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252400963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:401.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:401.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:401.602 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692252401539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43825 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:401.611 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:401.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:12:401.908 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:12:401.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:401.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692252401946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:401.991 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:401.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:402.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692252401953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:402.015 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:402.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:402.093 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692252402057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:12:402.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:402.132 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:402.148 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692252402110 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:12:402.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:402.694 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:402.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252402665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:402.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:402.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:402.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:12:402.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692252402706 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:12:402.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:12:450.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:450.981 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692252450881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:450.993 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:451.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:451.421 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692252451394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41801 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:451.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:12:451.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:12:451.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:12:451.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:12:451.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692252451770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:12:451.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:077.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:077.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692253077558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:077.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:078.359 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:078.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692253078315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44983 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:078.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:078.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:13:078.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:13:078.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:078.816 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692253078757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:078.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:369.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:369.983 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692253369878 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:369.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:370.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:370.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692253370395 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33901 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:370.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:370.621 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:13:370.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:13:370.773 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:370.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692253370760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:13:370.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:266.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:41266 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:14:275.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.18:8085::10.20.0.193:41280 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:14:717.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:717.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692254717280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:717.594 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:717.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:717.623 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692254717478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:717.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:717.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:14:717.916 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:14:717.978 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:718.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:14:718.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692254717968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:718.824 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:718.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:718.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692254717973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:718.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:718.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:718.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692254718060 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:14:718.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:718.904 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:718.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692254718128 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:14:718.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:718.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:718.948 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692254718705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:718.954 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:718.961 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:718.976 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692254718746 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:14:718.984 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:769.988 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:770.065 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692254769949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:770.080 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:770.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:770.218 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692254770135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:770.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:770.459 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:14:770.511 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:14:770.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:770.643 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692254770602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:770.654 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:914.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:915.078 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692254914853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:915.089 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:915.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:915.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692254915025 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43633 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:915.153 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:14:915.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:14:915.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:14:915.537 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:915.556 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692254915518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:14:915.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:034.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:034.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:034.651 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255034244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:034.660 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:034.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:034.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:034.797 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255034428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35555 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:034.806 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:034.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:034.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255034716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:034.831 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:034.837 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:034.846 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255034720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:034.852 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:034.859 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:034.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255034782 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:034.906 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:035.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:035.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255035079 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:035.131 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:035.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:035.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255035097 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:15:035.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:035.711 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:035.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692255035682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:035.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:035.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:035.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692255035738 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:15:035.792 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:15:035.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:15:081.029 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:081.098 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692255080988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:081.109 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:081.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:081.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692255081187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42169 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:081.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:081.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:081.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:081.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:081.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692255081666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:081.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:332.869 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:332.932 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255332833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:332.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:333.031 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:333.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692255333022 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41987 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:333.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:333.234 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:333.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:333.345 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:333.369 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692255333334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:333.377 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:787.800 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:787.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692255787756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:787.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:787.970 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:787.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692255787963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:787.994 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:15:788.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:15:788.350 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:15:788.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:15:788.423 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692255788387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:15:788.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:134.798 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:135.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692256134764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:135.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:135.126 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:135.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692256134946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:135.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:16:135.322 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:16:135.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:16:135.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:16:135.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692256135442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:16:135.492 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:606.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:606.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692257606529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:606.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:607.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:607.254 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692257607182 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:607.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:17:607.456 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:17:607.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:17:607.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:17:607.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692257607547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:17:607.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:187.674 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:187.735 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692258187638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:187.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:188.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:188.347 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692258188258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33699 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:188.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:18:188.557 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:18:188.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:18:188.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:18:188.692 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692258188644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:18:188.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:149.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:149.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692259149476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:149.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:150.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:150.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692259150156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46521 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:150.243 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:150.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:19:150.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:19:150.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:150.505 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259150458 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:150.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:150.528 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:150.544 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259150467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:150.554 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:150.572 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:150.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259150558 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:19:150.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:150.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:150.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259150606 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:19:150.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:151.156 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:151.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692259151104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:151.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:151.194 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:151.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692259151153 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:19:151.238 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:151.176 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:19:197.336 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:197.396 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692259197296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:197.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:197.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:197.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692259197829 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40955 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:197.887 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:198.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:19:198.265 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:19:198.319 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:198.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692259198296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:198.361 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:199.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:199.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692259199615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:199.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:199.917 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:199.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692259199910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33649 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:199.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:200.228 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:19:200.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:19:200.313 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:200.340 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259200301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:200.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:200.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:200.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259200306 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:200.381 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:200.388 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:200.418 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259200371 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:19:200.610 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:200.629 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:200.650 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259200462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:19:200.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:200.673 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:200.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692259200484 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:19:200.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:201.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:201.048 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692259200997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:201.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:201.063 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:201.073 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:19:201.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692259201038 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:19:201.122 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:19:248.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:248.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692259248201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:248.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:248.676 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:248.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692259248668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37673 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:248.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:248.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:19:248.987 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:19:249.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:249.075 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692259249033 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:249.083 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:116.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:116.763 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692260116456 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:116.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:116.793 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:116.811 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692260116640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33951 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:116.839 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:117.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:20:117.220 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:20:117.275 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:117.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692260117262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:117.315 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:266.279 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:266.349 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692260266239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:266.360 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:266.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:266.748 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692260266716 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:266.755 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:20:266.942 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:20:267.060 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:20:267.108 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:20:267.146 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692260267097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:20:267.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:207.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:207.753 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261207654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:207.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:207.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:207.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261207852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38275 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:207.965 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:208.161 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:21:208.210 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:21:208.269 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:209.034 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261208260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:209.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:209.068 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:209.081 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261208265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:209.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:209.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:209.134 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261208328 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:21:209.163 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:209.172 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:209.188 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261208382 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:21:209.197 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:209.208 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:209.223 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261208947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:209.233 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:209.241 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:209.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692261208990 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:21:209.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:209.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:21:262.187 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:262.251 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692261262155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:262.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:262.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:262.383 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692261262347 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44685 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:262.390 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:262.555 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:21:262.624 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:21:262.675 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:262.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692261262664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:262.712 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:860.804 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:860.868 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692261860773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:860.879 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:861.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:861.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692261861345 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42407 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:861.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:861.720 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:21:861.769 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:21:861.814 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:861.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692261861804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:861.843 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:861.850 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:861.860 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692261861811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:861.866 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:861.893 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:861.919 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692261861886 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:21:861.944 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:862.037 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:862.056 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692261861970 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:21:862.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:862.072 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:862.090 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692261861988 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:21:862.097 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:862.598 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:862.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692261862555 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:862.619 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:862.625 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:21:862.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:862.693 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692261862593 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:21:862.703 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:21:913.759 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:913.842 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692261913714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:913.857 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:913.918 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:913.947 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692261913906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40207 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:913.975 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:21:914.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:21:914.329 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:21:914.405 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:21:914.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692261914379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:21:914.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:085.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:085.413 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692262085279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:085.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:085.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:085.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692262085505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:085.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:085.737 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:085.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:085.865 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:085.884 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692262085846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:085.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:115.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:115.516 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692262115420 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:115.526 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:115.903 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:115.920 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692262115894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:115.934 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:116.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:116.365 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:116.410 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:116.438 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692262116399 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:116.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:270.873 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:270.936 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692262270838 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:271.155 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:271.165 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:271.178 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692262271032 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:271.185 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:271.258 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:271.308 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:271.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:271.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692262271360 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:271.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:877.646 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:877.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692262877614 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:877.731 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:878.159 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:878.182 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692262878148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:878.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:22:878.442 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:22:878.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:22:878.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:22:878.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692262878598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:22:878.636 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:305.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:305.921 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692264305563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:305.933 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:305.943 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:305.972 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:24:306.103 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692264305773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:306.102 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:24:306.125 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:306.286 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:306.321 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692264306217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:306.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:948.787 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:948.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692264948751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:948.867 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:949.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:949.431 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692264949353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33065 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:949.441 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:949.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:24:949.698 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:24:949.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:949.766 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692264949733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:949.772 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:758.079 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:758.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:758.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:758.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:758.550 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:758.666 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:758.683 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42427 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:758.691 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:758.699 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:758.709 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:758.715 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:758.721 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:758.743 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:758.749 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:758.756 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:758.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758665 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:758.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:758.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:758.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758958 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:758.999 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:759.007 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:759.023 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692265758984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:25:759.035 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:759.601 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:759.620 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692265759565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:759.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:759.638 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:759.671 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692265759606 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:759.681 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:759.627 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:25:788.563 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:788.631 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692265788528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:788.641 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:788.989 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:789.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692265788980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34841 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:789.012 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:789.355 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:789.445 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:790.276 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:25:790.375 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:790.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692265789507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:790.398 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:790.407 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:790.416 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692265789514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:790.422 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:790.429 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:790.453 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692265789583 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:790.475 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:790.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:790.501 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692265789646 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:25:790.508 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:790.515 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:790.525 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692265790208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:790.531 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:790.538 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:790.553 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692265790255 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:790.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:804.757 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:804.815 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692265804721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:804.825 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:805.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:805.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692265805297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:805.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:805.607 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:805.669 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:805.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:805.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692265805701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:805.762 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:839.403 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:839.466 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692265839369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:839.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:839.872 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:839.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692265839865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40143 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:839.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:840.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:840.299 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:840.357 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:840.378 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692265840332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:840.386 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:758.591 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:758.655 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692266758514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:758.665 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:758.877 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:758.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692266758756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43701 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:758.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:759.082 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:26:759.183 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:26:759.257 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:759.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692266759248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:759.289 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:805.371 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:805.428 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692266804890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:805.440 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:805.450 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:805.486 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692266805312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:805.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:26:805.664 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:26:805.779 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:26:805.835 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:805.855 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692266805816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:26:805.864 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:590.499 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:590.559 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692268590380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:590.581 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:591.009 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:591.033 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692268590988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:591.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:591.230 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:28:591.278 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:28:591.971 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:591.996 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692268591313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:592.005 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:592.014 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:592.024 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692268591319 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:592.030 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:592.036 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:592.058 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692268591389 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:28:592.076 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:28:592.084 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:592.101 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692268591441 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:28:592.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:28:592.129 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:592.139 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692268591938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:592.145 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:592.152 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:592.167 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692268591961 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:28:592.179 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:28:592.253 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:28:613.427 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:613.495 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692268613392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:613.506 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:613.579 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:613.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692268613569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35913 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:613.600 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:613.889 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:28:613.959 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:28:614.011 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:614.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692268614003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:614.045 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:640.498 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:640.668 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692268640463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:640.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:640.688 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:640.716 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692268640636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40127 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:640.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:641.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:28:641.198 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:28:641.246 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:641.285 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692268641229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:641.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:840.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:840.346 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692268840027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:840.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:840.368 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:840.380 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692268840240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35959 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:840.389 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:28:840.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:28:840.700 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:28:840.810 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:28:840.840 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692268840759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:28:840.848 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:125.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.092 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692269125875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:126.106 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:126.124 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.137 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692269126056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:126.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:126.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:126.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:126.536 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.723 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692269126527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:126.734 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:126.742 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.751 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692269126532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:126.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:126.765 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692269126607 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:126.832 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:126.892 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692269126879 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:126.940 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:126.949 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:126.966 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692269126899 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:29:126.982 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:127.545 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:127.561 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692269127507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:127.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:127.576 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:127.596 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692269127563 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:29:127.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:29:127.599 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:29:180.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:180.777 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692269180669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:180.788 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:180.912 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:180.945 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692269180904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:180.952 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:181.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:181.209 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:181.256 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:181.282 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692269181245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:181.290 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:604.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:604.568 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692269604464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:604.754 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:604.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:604.782 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692269604684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:604.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:29:605.022 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:29:605.088 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:29:605.217 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:29:605.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692269605164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:29:605.264 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:252.502 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:252.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692270252467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:252.582 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:253.028 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:253.057 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692270253014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:253.064 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:253.261 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:30:253.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:30:253.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:253.560 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692270253498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:253.569 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:585.424 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:585.512 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692271585376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:31:585.588 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:585.645 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:585.706 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692271585634 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:31:585.717 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:585.956 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:31:586.041 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:31:586.095 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:586.166 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692271586081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:31:586.177 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:31:829.630 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:829.714 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692271829467 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 1587 dequeues = 1587 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 1615 dequeues = 1615 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692242845330 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692271586091 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:04:31:829.729 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:31:829.738 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:829.896 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692271829591 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 317 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 361 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1213 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 347 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 795, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1587, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:31:829.928 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:31:838.107 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:838.180 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692271838077 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:04:31:838.191 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:04:31:838.200 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:838.327 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692271838145 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 in_tpkts = 487 in_bytes = 53576 out_tpkts = 525 out_bytes = 29409 exception_packets = 29 exception_packets_dropped = 11 raw_vhost_stats= [ name = vhost0 in_pkts = 487 in_bytes = 53576 out_pkts = 525 out_bytes = 29409 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 7.63278e-15 total_out_bandwidth_utilization = 1.68322e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1408, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3105, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 449 in_bytes = 26217 out_pkts = 490 out_bytes = 53702 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:31:838.354 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:31:838.363 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:31:838.391 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692271838180 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:31:838.399 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:32:143.722 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:143.801 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692272143686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:143.829 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:144.247 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:144.268 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692272144237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:144.277 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:144.485 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:144.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:144.670 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:144.689 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272144661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:144.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:801.004 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:801.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272800969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:801.297 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:801.310 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:801.323 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272801161 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32853 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:801.330 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:801.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:801.593 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:801.653 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.255 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272801642 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:802.270 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:802.280 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.294 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272801648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:802.300 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:802.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272801709 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:802.343 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:802.353 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.370 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272801831 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:802.387 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:802.395 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.411 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272801848 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:32:802.417 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:802.446 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.458 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272802423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:802.464 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:802.472 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:802.490 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272802465 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:802.503 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:802.509 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:32:845.447 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:32:845.830 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:04:32:845.973 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:32:846.010 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272845583 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:846.043 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:846.055 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:32:846.231 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:32:846.242 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:32:846.356 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272845953 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 321 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 363 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1221 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 349 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 800, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1604, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:32:846.385 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:32:846.397 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:32:846.404 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:32:846.527 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:846.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Idle 2025-09-01 Mon 02:04:32:846.626 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:846.809 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272846467 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 322 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 363 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1221 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 349 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 800, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1606, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:32:846.834 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:32:846.844 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:846.862 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272846500 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:846.875 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:846.882 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:32:846.910 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:847.158 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272846610 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 24715 http_port = 8085 start_time = 1756692121826560 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.249:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 322 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 364 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1221 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 349 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 801, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1606, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:32:847.181 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:32:852.574 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:852.609 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692272852557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:852.618 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:853.154 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:853.195 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272853130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44069 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:853.212 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:853.420 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:853.470 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:853.802 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:853.819 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692272853506 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:853.826 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:898.232 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:898.295 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692272898196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:898.543 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:898.552 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:898.565 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692272898385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:898.571 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:898.667 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:898.730 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:898.791 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:899.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-1/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-1/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:04:32:899.519 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272898783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:899.612 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:899.622 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:899.635 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272898788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:899.642 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:899.649 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:899.672 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272898845 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:899.684 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:899.696 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:899.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272898896 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:04:32:899.724 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:899.741 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:899.752 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272899426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:899.758 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:899.764 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:899.780 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272899470 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:32:899.790 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:32:948.640 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:948.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692272948602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:948.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:949.224 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:949.263 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692272949198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:949.273 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:949.408 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:949.469 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:949.530 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:949.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: SANDESH: Send FAILED: 1756692272949507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:949.613 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:858.634 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:858.702 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: SANDESH: Send FAILED: 1756692273858597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:858.713 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390963193344, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:859.111 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:859.143 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692273859103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43397 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:859.150 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:859.332 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:33:859.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390971590144, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:33:859.494 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:859.513 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: SANDESH: Send FAILED: 1756692273859473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:859.522 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390967391744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:924.605 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:924.679 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: SANDESH: Send FAILED: 1756692273924580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:924.690 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:925.013 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:925.039 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692273925005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34457 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:925.046 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:33:925.311 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:33:925.433 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390958994944, Pid 24715]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:33:925.491 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:33:925.532 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: SANDESH: Send FAILED: 1756692273925480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:33:925.542 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390946399744, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:510.307 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:510.372 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: SANDESH: Send FAILED: 1756692275510263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:510.382 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390975788544, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:510.894 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:35:510.937 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: SANDESH: Send FAILED: 1756692275510867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43059 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:35:510.946 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390950598144, Pid 24715]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:35:511.235 UTC an-jenkins-deploy-platform-ansible-os-3995-1 [Thread 140390982777472, Pid 24715]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299