2025-09-01 Mon 02:01:07:400.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: vhost interface name : vhost0 2025-09-01 Mon 02:01:07:400.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: vhost IP Address : 10.20.0.192/24 2025-09-01 Mon 02:01:07:400.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: vhost gateway : 0.0.0.0 2025-09-01 Mon 02:01:07:400.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Ethernet ports : ens6 2025-09-01 Mon 02:01:07:400.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Loopback IP : 0.0.0.0 2025-09-01 Mon 02:01:07:400.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Xmpp Servers : 10.20.0.25:5269 10.20.0.193:5269 10.20.0.129:5269 2025-09-01 Mon 02:01:07:400.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Xmpp Authentication : 0 2025-09-01 Mon 02:01:07:400.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Cluster-Name : 2025-09-01 Mon 02:01:07:400.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: DNS Servers : 10.20.0.25:53 10.20.0.193:53 10.20.0.129:53 2025-09-01 Mon 02:01:07:400.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: DNS client port : 8098 2025-09-01 Mon 02:01:07:400.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: DNS timeout : 3000 2025-09-01 Mon 02:01:07:400.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: DNS max retries : 2 2025-09-01 Mon 02:01:07:400.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Xmpp Dns Authentication : 0 2025-09-01 Mon 02:01:07:400.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TSN Servers : [] 2025-09-01 Mon 02:01:07:400.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: COLLECTOR Servers : 10.0.0.249:8086 10.0.0.38:8086 10.0.0.131:8086 2025-09-01 Mon 02:01:07:400.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Tunnel-Type : MPLSoGRE 2025-09-01 Mon 02:01:07:400.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Metadata-Proxy Shared Secret: contrail 2025-09-01 Mon 02:01:07:400.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Metadata-Proxy Port : 8097 2025-09-01 Mon 02:01:07:400.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Metadata-Proxy SSL Flag : 0 2025-09-01 Mon 02:01:07:400.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Linklocal Max System Flows : 16384 2025-09-01 Mon 02:01:07:400.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Linklocal Max Vm Flows : 16384 2025-09-01 Mon 02:01:07:400.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow cache timeout : 0 2025-09-01 Mon 02:01:07:400.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Stale Interface cleanup timeout : 60 2025-09-01 Mon 02:01:07:400.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow thread count : 1 2025-09-01 Mon 02:01:07:400.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow latency limit : 0 2025-09-01 Mon 02:01:07:400.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow index-mgr sm log count : 0 2025-09-01 Mon 02:01:07:400.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow add-tokens : 50 2025-09-01 Mon 02:01:07:400.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow ksync-tokens : 25 2025-09-01 Mon 02:01:07:400.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow del-tokens : 16 2025-09-01 Mon 02:01:07:400.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow update-tokens : 16 2025-09-01 Mon 02:01:07:400.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Pin flow netlink task to CPU: 2025-09-01 Mon 02:01:07:400.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Maximum sessions : 100 2025-09-01 Mon 02:01:07:400.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Maximum session aggregates : 8 2025-09-01 Mon 02:01:07:400.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Maximum session endpoints : 5 2025-09-01 Mon 02:01:07:400.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Fabric SNAT hash table size : 4096 2025-09-01 Mon 02:01:07:400.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Flow excluding Router ID in hash :0 2025-09-01 Mon 02:01:07:400.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Agent Mode : Vrouter 2025-09-01 Mon 02:01:07:400.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Gateway Mode : None 2025-09-01 Mon 02:01:07:400.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: DHCP Relay Mode : 0 2025-09-01 Mon 02:01:07:400.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Service instance netns cmd : /usr/local/bin/opencontrail-vrouter-netns 2025-09-01 Mon 02:01:07:400.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Service instance docker cmd : /usr/local/bin/opencontrail-vrouter-docker 2025-09-01 Mon 02:01:07:400.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Service instance workers : 0 2025-09-01 Mon 02:01:07:400.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Service instance timeout : 0 2025-09-01 Mon 02:01:07:400.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Service instance lb ssl : 2025-09-01 Mon 02:01:07:400.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Service instance lbaas auth : 2025-09-01 Mon 02:01:07:400.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Bgp as a service port range : 50000-50512 2025-09-01 Mon 02:01:07:400.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Services queue limit : 1024 2025-09-01 Mon 02:01:07:400.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: BGPAAS max shared sessions for service port : 4 2025-09-01 Mon 02:01:07:400.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Sandesh Key file : /etc/contrail/ssl/private/server-privkey.pem 2025-09-01 Mon 02:01:07:400.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Sandesh Cert file : /etc/contrail/ssl/certs/server.pem 2025-09-01 Mon 02:01:07:400.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Sandesh CA Cert : /etc/contrail/ssl/certs/ca-cert.pem 2025-09-01 Mon 02:01:07:400.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Sandesh SSL Enable : 0 2025-09-01 Mon 02:01:07:400.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Introspect SSL Enable : 0 2025-09-01 Mon 02:01:07:400.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Hypervisor mode : kvm 2025-09-01 Mon 02:01:07:400.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Ethernet Port Encap Type : 2025-09-01 Mon 02:01:07:400.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: Platform mode : Vrouter on host linux kernel 2025-09-01 Mon 02:01:07:400.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: VRouter [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-09-01 Mon 02:01:07:400.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Logging: DISABLED -> ENABLED 2025-09-01 Mon 02:01:07:400.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-09-01 Mon 02:01:07:415.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> self 2025-09-01 Mon 02:01:07:415.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:01:07:415.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:01:07:415.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-access-control-list -> self 2025-09-01 Mon 02:01:07:415.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap access-control-list : self -> virtual-network-access-control-list 2025-09-01 Mon 02:01:07:415.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap access-control-list : virtual-network-access-control-list -> nil 2025-09-01 Mon 02:01:07:415.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> self 2025-09-01 Mon 02:01:07:415.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:01:07:415.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> virtual-network-network-ipam 2025-09-01 Mon 02:01:07:415.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:01:07:415.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-qos-config -> self 2025-09-01 Mon 02:01:07:415.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-config : self -> virtual-network-qos-config 2025-09-01 Mon 02:01:07:415.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-config : virtual-network-qos-config -> nil 2025-09-01 Mon 02:01:07:415.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-security-logging-object -> self 2025-09-01 Mon 02:01:07:415.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : self -> virtual-network-security-logging-object 2025-09-01 Mon 02:01:07:415.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : virtual-network-security-logging-object -> nil 2025-09-01 Mon 02:01:07:415.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : logical-router-virtual-network -> self 2025-09-01 Mon 02:01:07:415.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-router-virtual-network : self -> logical-router-virtual-network 2025-09-01 Mon 02:01:07:415.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-router-virtual-network : logical-router-virtual-network -> logical-router-virtual-network 2025-09-01 Mon 02:01:07:415.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-router : self -> logical-router-virtual-network 2025-09-01 Mon 02:01:07:415.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-router : logical-router-virtual-network -> nil 2025-09-01 Mon 02:01:07:415.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:01:07:415.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-multicast-policy -> self 2025-09-01 Mon 02:01:07:415.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap multicast-policy : self -> virtual-network-multicast-policy 2025-09-01 Mon 02:01:07:415.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap multicast-policy : virtual-network-multicast-policy -> nil 2025-09-01 Mon 02:01:07:415.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for routing-instance 2025-09-01 Mon 02:01:07:415.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> self 2025-09-01 Mon 02:01:07:415.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-routing-instance 2025-09-01 Mon 02:01:07:415.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-routing-instance 2025-09-01 Mon 02:01:07:415.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:01:07:415.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:01:07:415.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for security-group 2025-09-01 Mon 02:01:07:415.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-group : security-group-access-control-list -> self 2025-09-01 Mon 02:01:07:415.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap access-control-list : self -> security-group-access-control-list 2025-09-01 Mon 02:01:07:415.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap access-control-list : security-group-access-control-list -> nil 2025-09-01 Mon 02:01:07:415.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for tag 2025-09-01 Mon 02:01:07:415.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : application-policy-set-tag -> self 2025-09-01 Mon 02:01:07:415.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:01:07:415.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : policy-management-application-policy-set -> application-policy-set-tag 2025-09-01 Mon 02:01:07:415.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap policy-management : policy-management-application-policy-set -> nil 2025-09-01 Mon 02:01:07:415.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:01:07:415.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:415.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:415.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:01:07:415.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:01:07:415.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:01:07:415.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:01:07:415.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine : virtual-machine-interface-virtual-machine -> nil 2025-09-01 Mon 02:01:07:415.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:01:07:415.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:01:07:415.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> nil 2025-09-01 Mon 02:01:07:415.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:01:07:415.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:415.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:415.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:01:07:415.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:01:07:415.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-routing-instance -> self 2025-09-01 Mon 02:01:07:415.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface-routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:01:07:415.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface-routing-instance : virtual-machine-interface-routing-instance -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:01:07:415.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:01:07:415.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : virtual-machine-interface-routing-instance -> nil 2025-09-01 Mon 02:01:07:415.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-group -> self 2025-09-01 Mon 02:01:07:415.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-group : self -> virtual-machine-interface-security-group 2025-09-01 Mon 02:01:07:415.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-group : virtual-machine-interface-security-group -> nil 2025-09-01 Mon 02:01:07:415.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : alias-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:415.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap alias-ip : self -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap alias-ip : alias-ip-pool-alias-ip -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap alias-ip-pool : virtual-network-alias-ip-pool -> alias-ip-pool-alias-ip 2025-09-01 Mon 02:01:07:415.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-alias-ip-pool 2025-09-01 Mon 02:01:07:415.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-alias-ip-pool -> nil 2025-09-01 Mon 02:01:07:415.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:415.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:01:07:415.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:01:07:415.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:01:07:415.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:01:07:415.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:01:07:415.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:415.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:01:07:415.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:01:07:415.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:01:07:415.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:01:07:415.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:01:07:415.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:415.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap floating-ip : instance-ip-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap instance-ip : instance-ip-virtual-network -> instance-ip-floating-ip 2025-09-01 Mon 02:01:07:415.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> instance-ip-virtual-network 2025-09-01 Mon 02:01:07:415.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : instance-ip-virtual-network -> nil 2025-09-01 Mon 02:01:07:415.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : instance-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:415.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap instance-ip : self -> instance-ip-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap instance-ip : instance-ip-virtual-machine-interface -> nil 2025-09-01 Mon 02:01:07:415.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-route-table -> self 2025-09-01 Mon 02:01:07:415.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap interface-route-table : self -> virtual-machine-interface-route-table 2025-09-01 Mon 02:01:07:415.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap interface-route-table : virtual-machine-interface-route-table -> nil 2025-09-01 Mon 02:01:07:415.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:415.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : subnet-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:415.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap subnet : self -> subnet-virtual-machine-interface 2025-09-01 Mon 02:01:07:415.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap subnet : subnet-virtual-machine-interface -> nil 2025-09-01 Mon 02:01:07:415.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:416.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:01:07:416.017 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:01:07:416.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:01:07:416.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-port-group-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:416.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> virtual-port-group-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : physical-router-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:01:07:416.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:01:07:416.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:01:07:416.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:416.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-as-a-service : bgpaas-bgp-router -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-router : self -> bgpaas-bgp-router 2025-09-01 Mon 02:01:07:416.097 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-router : instance-bgp-router -> bgpaas-bgp-router 2025-09-01 Mon 02:01:07:416.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : self -> instance-bgp-router 2025-09-01 Mon 02:01:07:416.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : instance-bgp-router -> nil 2025-09-01 Mon 02:01:07:416.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:416.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-as-a-service : bgpaas-health-check -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap service-health-check : self -> bgpaas-health-check 2025-09-01 Mon 02:01:07:416.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap service-health-check : bgpaas-health-check -> nil 2025-09-01 Mon 02:01:07:416.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-qos-config -> self 2025-09-01 Mon 02:01:07:416.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-config : self -> virtual-machine-interface-qos-config 2025-09-01 Mon 02:01:07:416.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-config : virtual-machine-interface-qos-config -> nil 2025-09-01 Mon 02:01:07:416.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-logging-object -> self 2025-09-01 Mon 02:01:07:416.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : self -> virtual-machine-interface-security-logging-object 2025-09-01 Mon 02:01:07:416.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : virtual-machine-interface-security-logging-object -> nil 2025-09-01 Mon 02:01:07:416.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:01:07:416.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:01:07:416.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:01:07:416.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:01:07:416.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:01:07:416.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bridge-domain : virtual-machine-interface-bridge-domain -> nil 2025-09-01 Mon 02:01:07:416.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-tag -> self 2025-09-01 Mon 02:01:07:416.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> virtual-machine-interface-tag 2025-09-01 Mon 02:01:07:416.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-interface-tag 2025-09-01 Mon 02:01:07:416.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:01:07:416.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:01:07:416.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:416.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-tag -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:416.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> virtual-network-tag 2025-09-01 Mon 02:01:07:416.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : application-policy-set-tag -> virtual-network-tag 2025-09-01 Mon 02:01:07:416.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:01:07:416.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:01:07:416.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:01:07:416.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:01:07:416.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:01:07:416.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:01:07:416.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:01:07:416.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:01:07:416.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:01:07:416.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:01:07:416.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:01:07:416.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:01:07:416.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:01:07:416.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:01:07:416.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:01:07:416.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : project-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:416.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap project : self -> project-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap project : project-tag -> project-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> project-tag 2025-09-01 Mon 02:01:07:416.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : application-policy-set-tag -> project-tag 2025-09-01 Mon 02:01:07:416.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:01:07:416.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:416.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:01:07:416.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> self 2025-09-01 Mon 02:01:07:416.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> nil 2025-09-01 Mon 02:01:07:416.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:01:07:416.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : logical-router-interface -> self 2025-09-01 Mon 02:01:07:416.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-router : self -> logical-router-interface 2025-09-01 Mon 02:01:07:416.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-router : logical-router-interface -> nil 2025-09-01 Mon 02:01:07:416.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:01:07:416.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : physical-router-physical-interface -> self 2025-09-01 Mon 02:01:07:416.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:01:07:416.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:01:07:416.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:01:07:416.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : virtual-port-group-physical-interface -> self 2025-09-01 Mon 02:01:07:416.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-port-group-physical-interface : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:01:07:416.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-port-group-physical-interface : virtual-port-group-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:01:07:416.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-port-group : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:01:07:416.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> nil 2025-09-01 Mon 02:01:07:416.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:01:07:416.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> self 2025-09-01 Mon 02:01:07:416.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : self -> physical-interface-logical-interface 2025-09-01 Mon 02:01:07:416.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:01:07:416.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:01:07:416.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:01:07:416.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:01:07:416.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:01:07:416.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:01:07:416.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> nil 2025-09-01 Mon 02:01:07:416.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:01:07:416.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap logical-interface : physical-router-logical-interface -> self 2025-09-01 Mon 02:01:07:416.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : self -> physical-router-logical-interface 2025-09-01 Mon 02:01:07:416.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : physical-router-logical-interface -> nil 2025-09-01 Mon 02:01:07:416.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for physical-router 2025-09-01 Mon 02:01:07:416.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-router : physical-router-physical-interface -> self 2025-09-01 Mon 02:01:07:416.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : self -> physical-router-physical-interface 2025-09-01 Mon 02:01:07:416.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap physical-interface : physical-router-physical-interface -> nil 2025-09-01 Mon 02:01:07:416.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:01:07:416.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:01:07:416.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:01:07:416.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> service-port-health-check 2025-09-01 Mon 02:01:07:416.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:416.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:01:07:416.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:01:07:416.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:01:07:416.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:01:07:416.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:01:07:416.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> service-port-health-check 2025-09-01 Mon 02:01:07:416.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> port-tuple-interface 2025-09-01 Mon 02:01:07:416.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:01:07:416.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:01:07:416.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:01:07:416.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for qos-config 2025-09-01 Mon 02:01:07:416.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-config : global-qos-config-qos-config -> self 2025-09-01 Mon 02:01:07:416.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap global-qos-config : global-qos-config-qos-config -> nil 2025-09-01 Mon 02:01:07:416.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for forwarding-class 2025-09-01 Mon 02:01:07:416.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap forwarding-class : forwarding-class-qos-queue -> self 2025-09-01 Mon 02:01:07:416.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-queue : self -> forwarding-class-qos-queue 2025-09-01 Mon 02:01:07:416.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap qos-queue : forwarding-class-qos-queue -> nil 2025-09-01 Mon 02:01:07:416.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:01:07:416.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : security-logging-object-network-policy -> self 2025-09-01 Mon 02:01:07:416.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap network-policy : self -> security-logging-object-network-policy 2025-09-01 Mon 02:01:07:416.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap network-policy : security-logging-object-network-policy -> nil 2025-09-01 Mon 02:01:07:416.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:01:07:416.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : security-logging-object-security-group -> self 2025-09-01 Mon 02:01:07:416.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-group : self -> security-logging-object-security-group 2025-09-01 Mon 02:01:07:416.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-group : security-logging-object-security-group -> nil 2025-09-01 Mon 02:01:07:416.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:01:07:416.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : firewall-policy-security-logging-object -> self 2025-09-01 Mon 02:01:07:416.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-security-logging-object : self -> firewall-policy-security-logging-object 2025-09-01 Mon 02:01:07:416.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-security-logging-object : firewall-policy-security-logging-object -> firewall-policy-security-logging-object 2025-09-01 Mon 02:01:07:416.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : firewall-policy-security-logging-object -> nil 2025-09-01 Mon 02:01:07:416.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:01:07:416.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap security-logging-object : firewall-rule-security-logging-object -> self 2025-09-01 Mon 02:01:07:416.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule-security-logging-object : self -> firewall-rule-security-logging-object 2025-09-01 Mon 02:01:07:416.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule-security-logging-object : firewall-rule-security-logging-object -> firewall-rule-security-logging-object 2025-09-01 Mon 02:01:07:416.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : firewall-rule-security-logging-object -> nil 2025-09-01 Mon 02:01:07:416.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for virtual-router 2025-09-01 Mon 02:01:07:416.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-router : virtual-router-sub-cluster -> self 2025-09-01 Mon 02:01:07:417.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap sub-cluster : self -> virtual-router-sub-cluster 2025-09-01 Mon 02:01:07:417.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap sub-cluster : bgp-router-sub-cluster -> virtual-router-sub-cluster 2025-09-01 Mon 02:01:07:417.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-router : self -> bgp-router-sub-cluster 2025-09-01 Mon 02:01:07:417.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bgp-router : bgp-router-sub-cluster -> nil 2025-09-01 Mon 02:01:07:417.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:01:07:417.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:01:07:417.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:01:07:417.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-bridge-domain 2025-09-01 Mon 02:01:07:417.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:01:07:417.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:01:07:417.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:01:07:417.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:01:07:417.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:01:07:417.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:01:07:417.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for application-policy-set 2025-09-01 Mon 02:01:07:417.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> self 2025-09-01 Mon 02:01:07:417.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:417.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:417.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:01:07:417.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:01:07:417.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:01:07:417.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:01:07:417.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : firewall-rule-service-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap service-group : self -> firewall-rule-service-group 2025-09-01 Mon 02:01:07:417.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap service-group : firewall-rule-service-group -> nil 2025-09-01 Mon 02:01:07:417.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:01:07:417.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:01:07:417.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : firewall-rule-address-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap address-group : self -> firewall-rule-address-group 2025-09-01 Mon 02:01:07:417.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap address-group : address-group-tag -> firewall-rule-address-group 2025-09-01 Mon 02:01:07:417.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> address-group-tag 2025-09-01 Mon 02:01:07:417.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : address-group-tag -> nil 2025-09-01 Mon 02:01:07:417.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:01:07:417.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:01:07:417.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : firewall-rule-tag -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : self -> firewall-rule-tag 2025-09-01 Mon 02:01:07:417.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap tag : firewall-rule-tag -> nil 2025-09-01 Mon 02:01:07:417.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:01:07:417.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:01:07:417.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:01:07:417.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Adding ReactorMap firewall-rule : firewall-policy-firewall-rule -> nil 2025-09-01 Mon 02:01:07:545.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.192:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:01:07:549.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Current receive sock buffer size is 262144 2025-09-01 Mon 02:01:07:549.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691738297856, Pid 23614]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU 2025-09-01 Mon 02:01:07:552.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Vrouter family is 34 2025-09-01 Mon 02:01:07:552.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Flow table size : 20185088 2025-09-01 Mon 02:01:07:552.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Flow table size : 161218560 2025-09-01 Mon 02:01:07:552.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SetHugePages: Bridge table size:20185088 Flow table size:161218560 2025-09-01 Mon 02:01:07:552.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Mem mapping hugepage file:/dev/hugepages/bridge size:20185088 2025-09-01 Mon 02:01:07:552.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Mem mapped hugepage file:/dev/hugepages/bridge to addr:0x7f0c82600000 2025-09-01 Mon 02:01:07:552.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Mem mapping hugepage file:/dev/hugepages/flow size:161218560 2025-09-01 Mon 02:01:07:552.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Mem mapped hugepage file:/dev/hugepages/flow to addr:0x7f0c78c00000 2025-09-01 Mon 02:01:07:552.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Sending Huge Page configuration to VROUTER 2025-09-01 Mon 02:01:07:603.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Huge pages set successfully 2025-09-01 Mon 02:01:07:615.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Flow table size : 161218560 2025-09-01 Mon 02:01:07:616.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Flow table size : 20185088 2025-09-01 Mon 02:01:07:656.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: ROLE : Generator 2025-09-01 Mon 02:01:07:656.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: MODULE : contrail-vrouter-agent 2025-09-01 Mon 02:01:07:656.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: SOURCE : an-jenkins-deploy-platform-ansible-os-3995-2 2025-09-01 Mon 02:01:07:656.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: NODE TYPE : Compute 2025-09-01 Mon 02:01:07:656.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: INSTANCE ID : 0 2025-09-01 Mon 02:01:07:656.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: HTTP SERVER PORT : 8085 2025-09-01 Mon 02:01:07:656.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-09-01 Mon 02:01:07:657.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: HTTP Introspect Init 2025-09-01 Mon 02:01:07:657.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8085 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:01:07:657.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Sandesh Http Server Port: 8085 2025-09-01 Mon 02:01:07:657.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: NOT Writing http_port 8085TO : /tmp/contrail-vrouter-agent.22999.http_port 2025-09-01 Mon 02:01:07:657.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: CONNECT TO COLLECTOR: 1 2025-09-01 Mon 02:01:07:658.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Idle 2025-09-01 Mon 02:01:07:659.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: No Client: 1756692067659393 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:07:660.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvStart in state Idle 2025-09-01 Mon 02:01:07:660.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Disconnect 2025-09-01 Mon 02:01:07:660.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvCollectorUpdate in state Disconnect 2025-09-01 Mon 02:01:07:661.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:01:07:661.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:07:661.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067660702 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:07:661.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:661.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:07:661.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067660764 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:07:661.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:07:661.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:07:661.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067660807 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:07:661.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:07:661.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:07:661.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067660824 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:07:661.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:661.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:07:661.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067661133 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:07:661.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:07:662.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:07:662.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:07:662.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:07:664.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067662000 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:01:07:664.595 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:01:07:664.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:07:664.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:07:665.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:07:665.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Idle 2025-09-01 Mon 02:01:07:665.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:665.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067662986 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:01:07:665.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:01:07:665.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:665.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067664895 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:07:665.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:07:665.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:665.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067665045 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:07:665.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:665.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:07:665.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface lo flags 65609 2025-09-01 Mon 02:01:07:665.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface ens3 flags 69699 2025-09-01 Mon 02:01:07:665.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface ens6 flags 69699 2025-09-01 Mon 02:01:07:665.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface docker0 flags 4099 2025-09-01 Mon 02:01:07:665.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface pkt1 flags 65601 2025-09-01 Mon 02:01:07:665.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface pkt3 flags 65601 2025-09-01 Mon 02:01:07:665.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface pkt2 flags 65601 2025-09-01 Mon 02:01:07:665.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface vhost0 flags 69699 2025-09-01 Mon 02:01:07:665.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink interface message RTM_NEWLINK for interface pkt0 flags 69699 2025-09-01 Mon 02:01:07:666.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:666.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067665181 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:07:666.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:07:666.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:01:07:666.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:01:07:666.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface docker0 2025-09-01 Mon 02:01:07:666.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:01:07:666.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:01:07:666.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:01:07:666.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:01:07:666.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink address message RTM_NEWADDR for interface pkt0 2025-09-01 Mon 02:01:07:666.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:01:07:666.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 10.0.0.0/24 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 10.0.0.1/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 10.0.0.2/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 10.0.0.4/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:01:07:666.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Handle netlink route message RTM_NEWROUTE : 172.17.0.0/16 Interface docker0 GW 0.0.0.0 2025-09-01 Mon 02:01:07:666.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:01:07:666.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:01:07:666.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:01:07:666.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:01:07:666.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:01:07:666.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:01:07:666.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:01:07:666.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:01:07:666.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:01:07:666.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:01:07:666.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:01:07:666.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:01:07:666.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:01:07:666.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:01:07:666.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:01:07:666.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:01:07:666.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:01:07:666.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:01:07:666.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:01:07:687.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:9091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:01:07:688.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Router ID configured. Connection to controller initiated 2025-09-01 Mon 02:01:07:693.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:693.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067693096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:693.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:694.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:694.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067694051 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, Collector connection down, No Configuration for self Number of connections:2, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:07:694.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:696.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:696.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067695390 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:696.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:696.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:697.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067696182 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, Collector connection down, No Configuration for self Number of connections:3, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:07:697.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:698.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:698.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067697424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:698.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:698.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:698.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067698220 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:07:698.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:700.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:700.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067699469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:700.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:701.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:701.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067700334 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:07:701.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:07:703.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpServerMessageLog: Server TCP bind(fe80::f816:3eff:fe63:a394%vhost0:8097): Cannot assign requested address src/contrail-common/io/tcp_server.cc 110 2025-09-01 Mon 02:01:07:751.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:751.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067751294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:751.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:751.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:752.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067751751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:752.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:753.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:07:753.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:07:753.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:753.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067753792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:753.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:753.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:753.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692067753873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:753.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:753.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:753.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692067753951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:753.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:755.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:07:755.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:07:755.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:755.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692067755226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:755.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:755.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:755.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692067755279 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:755.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:755.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:755.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692067755397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:755.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:755.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:07:755.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:07:755.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:755.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692067755681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:755.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:757.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:757.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067757240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:757.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:07:757.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:07:757.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692067757322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33709 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:07:757.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:665.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:08:666.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:01:08:667.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:08:667.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692068665240 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:08:667.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:08:667.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:08:667.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692068666462 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 10, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 29, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:08:667.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:08:668.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:08:668.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:08:668.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:08:668.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:08:669.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:08:669.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Idle 2025-09-01 Mon 02:01:08:669.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:669.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692068668336 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 10, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:08:669.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:08:669.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:669.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692068668369 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:08:669.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:08:669.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:08:669.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:669.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692068669408 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 11, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:08:669.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:08:685.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:685.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692068685393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvConnectTimerExpired close_reason = Connect timer expired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:685.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:744.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:744.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692068743892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:744.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:744.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:744.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692068744365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:744.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:744.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:08:744.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:08:744.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:744.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692068744788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:744.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:781.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:781.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692068781348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:781.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:781.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:781.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692068781795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:781.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:781.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:781.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692068781851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:781.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:781.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:781.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692068781935 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:781.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:782.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:08:782.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:08:782.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:08:782.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:782.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692068782325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:782.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:08:782.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:08:782.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:08:782.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692068782445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:08:782.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:09:666.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:09:667.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692069666804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:09:667.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:09:667.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:09:667.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692069666973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40709 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:09:667.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:09:667.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:09:667.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:09:667.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:09:667.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692069667370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:09:667.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:617.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:617.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692070616882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:617.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:617.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:617.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692070617898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40071 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:617.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:618.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:10:618.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:10:618.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:618.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692070618198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:618.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:669.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:10:671.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:01:10:671.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:10:671.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692070669574 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:10:671.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:10:671.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:10:671.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:10:671.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:10:671.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692070671383 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 21, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 50, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:10:671.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:10:671.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:10:671.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:10:671.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:10:671.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Idle 2025-09-01 Mon 02:01:10:671.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:672.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692070671814 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 21, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:10:672.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:10:672.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:672.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692070671855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:10:672.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:10:672.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:10:672.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:672.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692070671975 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 22, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:10:672.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:10:780.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:780.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692070780583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:780.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:781.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:781.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692070781532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:781.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:781.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:10:781.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:10:782.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:782.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692070781956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:782.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:852.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:852.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692070852473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:852.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:853.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:853.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692070853140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41153 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:853.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:10:853.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:10:853.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:10:853.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:10:853.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692070853583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:10:853.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:11:809.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:11:809.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692071809501 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:11:809.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:11:809.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:11:810.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692071809974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33377 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:11:810.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:11:810.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:11:810.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:11:810.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:11:810.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692071810567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:11:810.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:11:822.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:39196 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:14:290.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:290.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692074290372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:290.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:290.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:290.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692074290775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:290.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:291.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:14:291.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:291.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:291.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074291236 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:291.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:291.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:291.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074291242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:291.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:291.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:291.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074291296 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:291.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:292.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:292.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074292516 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:14:292.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:293.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:293.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074293007 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:293.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:293.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:293.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:14:293.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074293044 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:293.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:340.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:340.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074340486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:340.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:340.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:340.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074340662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:340.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:342.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:14:342.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:342.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:342.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692074342428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:342.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:418.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:418.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692074418058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:418.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:418.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:418.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692074418964 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45031 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:419.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:419.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:14:419.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:419.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:419.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074419403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:419.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:419.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:419.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074419407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:419.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:420.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:14:420.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:420.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074419522 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:420.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:420.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:420.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074419570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:14:420.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:420.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:420.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074420047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:420.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:420.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:420.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692074420072 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:420.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:471.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:471.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692074470944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:471.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:471.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:471.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074471924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:471.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:472.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:14:472.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:473.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:473.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074472992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:473.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:671.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:14:673.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:01:14:673.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:14:673.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074672103 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:673.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:673.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:14:673.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:14:673.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:14:673.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074673407 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 39, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 86, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:14:673.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:14:673.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:14:673.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:14:673.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:673.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Idle 2025-09-01 Mon 02:01:14:673.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:673.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074673763 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 39, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:14:673.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:14:673.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:674.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074673787 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:674.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:674.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:14:674.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:674.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074673862 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 40, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:14:674.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:14:885.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:885.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692074885728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:885.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:886.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:886.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692074886017 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:886.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:886.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:14:886.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:886.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:886.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692074886516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:886.552 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:886.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:886.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692074886521 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:886.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:886.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:886.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692074886619 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:886.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:887.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:887.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074887381 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:887.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:887.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:887.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074887401 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:14:887.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:887.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:887.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074887879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:887.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:887.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:887.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:14:887.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074887910 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:888.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:933.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:933.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692074933225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:933.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:933.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:933.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692074933776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:933.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:934.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:14:934.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:934.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:934.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692074934183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:14:934.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:314.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:314.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692075314575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:314.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:315.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:315.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692075315386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:315.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:315.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:15:315.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:15:315.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:315.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692075315827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:315.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:499.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:499.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692075499141 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:499.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:499.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:499.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692075499703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:499.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:500.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:15:500.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:15:500.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:500.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692075500127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:500.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:942.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:942.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692075942356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:942.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:942.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:942.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692075942515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39671 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:942.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:15:942.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:15:942.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:15:942.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:15:942.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692075942915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:15:942.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:059.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:059.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692076059719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:059.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:060.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:060.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:16:061.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:16:062.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:16:062.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076060462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:062.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:062.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076061235 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:062.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:062.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076061242 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:062.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:062.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076061333 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:16:062.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:16:062.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076061421 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:16:062.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:16:062.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076061438 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:16:062.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:16:062.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076062037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:062.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:062.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:062.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076062076 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:16:062.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:16:114.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:114.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692076114245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:114.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:114.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:114.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692076114410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:114.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:16:115.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:16:115.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:16:115.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:16:115.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692076115229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:16:115.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:060.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:060.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692077060393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:060.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:060.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:060.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692077060566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46211 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:060.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:060.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:17:061.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:17:061.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:061.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692077061097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:061.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:368.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:368.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692077368026 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:368.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:368.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:368.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692077368202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36371 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:368.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:368.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:17:368.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:17:368.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:368.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692077368662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:368.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:444.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:444.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:17:444.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:17:444.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692077444297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:444.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:444.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:444.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692077444455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45925 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:444.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:444.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:444.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692077444827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:444.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:905.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:905.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692077905076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:905.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:905.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:905.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692077905245 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:905.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:17:905.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:17:905.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:17:905.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:17:905.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692077905681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:17:905.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:18:933.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:18:933.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692078933170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:18:933.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:18:934.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:18:934.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692078934060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39571 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:18:934.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:18:934.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:18:934.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:18:934.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:18:934.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692078934430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:18:934.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:261.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:261.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692081260974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:261.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:261.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:261.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692081261184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:261.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:261.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:21:261.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:21:261.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:261.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692081261549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:261.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:261.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:261.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692081261555 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:261.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:261.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:261.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692081261649 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:21:261.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:21:261.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:261.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692081261696 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:21:261.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:21:262.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:262.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692081262238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:262.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:262.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:262.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692081262268 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:21:262.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:21:262.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:21:310.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:310.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692081310453 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:310.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:311.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:311.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692081311068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:311.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:311.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:21:311.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:21:311.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:311.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692081311537 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:311.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:472.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:473.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692081472822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:473.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:473.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:473.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692081473009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36909 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:473.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:473.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:21:473.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:21:473.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:473.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692081473409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:473.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:473.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:473.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692081473415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:473.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:473.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:473.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692081473497 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:21:473.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:21:473.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:473.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692081473539 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:21:473.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:21:474.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:21:474.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:474.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692081474216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:474.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:474.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:474.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692081474257 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:21:474.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:21:525.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:525.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692081525430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:525.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:525.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:525.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692081525620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38691 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:525.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:21:525.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:21:525.915 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:21:525.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:21:526.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692081525962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:21:526.007 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:189.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:190.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692082189810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:190.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:190.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:190.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692082190518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:190.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:190.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:22:191.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:22:191.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692082191052 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:191.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:191.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692082191058 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:191.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:191.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692082191108 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:22:191.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:22:191.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692082191209 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:22:191.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:22:191.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692082191232 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:22:191.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:22:191.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692082191758 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:191.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:191.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:191.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692082191789 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:22:191.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:22:191.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:22:229.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:229.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692082229682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:229.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:230.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:230.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692082230372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35057 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:230.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:230.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:22:230.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:22:230.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:230.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692082230771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:230.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:238.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:238.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692082237983 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:238.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:239.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:239.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692082239103 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:239.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:239.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:22:239.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:22:239.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:239.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692082239489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:239.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:543.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:543.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692082543070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:543.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:544.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:544.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692082544067 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35565 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:544.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:544.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:22:544.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:22:544.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:544.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692082544597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:22:544.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:22:673.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:22:675.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:01:22:675.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:22:675.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692082673991 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:22:675.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:22:675.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:22:675.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:22:675.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:22:675.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692082675446 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 126 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 179, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:22:675.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:22:675.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:22:675.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:22:675.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:22:675.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Idle 2025-09-01 Mon 02:01:22:675.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:676.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692082675772 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 126 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 181, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:22:676.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:22:676.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:676.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692082675811 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:22:676.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:22:676.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:22:676.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:22:676.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692082675934 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 41 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 126 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 181, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:22:676.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:23:038.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:038.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692083038539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:038.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:038.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:039.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692083038985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33445 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:039.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:039.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:23:039.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:23:039.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:039.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692083039596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:039.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:039.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:039.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692083039601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:039.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:039.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:039.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692083039653 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:23:039.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:23:039.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:039.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692083039741 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:23:039.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:23:039.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:039.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692083039757 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:23:039.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:23:040.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:040.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692083040301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:040.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:040.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:040.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692083040334 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:23:040.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:23:040.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:23:090.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:090.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692083090597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:090.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:091.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:091.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692083091194 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36779 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:091.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:091.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:23:091.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:23:091.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:091.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692083091557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:091.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:193.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:193.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692083193618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:193.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:194.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:194.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692083194346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:194.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:23:194.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:23:194.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:23:194.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:23:194.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692083194750 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:23:194.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:090.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:090.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692084090696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:090.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:091.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:091.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692084091613 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33179 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:091.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:091.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:24:092.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:24:092.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:092.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692084092090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:092.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:103.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:103.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692084102894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:103.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:104.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:104.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692084103988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:104.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:104.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:24:104.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:24:104.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:104.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692084104548 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:104.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:362.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:362.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692084362778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:362.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:362.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:362.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692084362951 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43475 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:362.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:24:363.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:24:363.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:24:363.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:24:363.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692084363353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:24:363.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:25:102.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:25:103.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692085102890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:25:103.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:25:103.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:25:103.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692085103376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:25:103.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:25:103.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:25:103.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:25:103.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:25:103.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692085103765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:25:103.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:25:964.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:25:964.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692085964241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:25:964.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:25:965.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:25:965.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692085965148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46743 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:25:965.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:25:965.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:25:965.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:25:965.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:25:965.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692085965558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:25:965.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:992.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:992.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692087992671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:992.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:993.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:993.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692087993171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:993.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:993.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:27:993.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:27:993.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:993.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692087993647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:993.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:993.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:993.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692087993652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:993.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:993.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:993.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692087993700 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:27:993.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:27:993.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:993.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692087993750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:27:993.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:27:994.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:994.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692087994285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:994.361 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:994.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:994.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692087994321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:27:994.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:27:994.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:28:046.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:046.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692088046787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:046.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:047.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:047.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692088047336 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:047.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:047.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:28:047.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:28:047.726 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:047.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692088047715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:047.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:143.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:143.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692088143496 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:143.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:144.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:144.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692088144157 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44091 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:144.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:144.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:28:144.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:28:144.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:145.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692088144599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:145.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:145.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:145.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692088144607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:145.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:145.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:145.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692088144663 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:28:145.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:28:145.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:145.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692088144707 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:28:145.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:28:145.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:145.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692088145190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:145.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:145.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:145.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692088145230 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:28:145.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:28:145.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:28:195.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:195.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692088195402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:195.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:195.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:195.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692088195920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:195.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:28:196.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:28:196.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:28:196.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:28:196.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692088196543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:28:196.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:028.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:028.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692089027894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:028.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:028.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:028.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089028644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:028.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:029.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:029.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:029.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:029.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089029254 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:029.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:029.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:029.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089029259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:029.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:029.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:029.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089029306 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:29:029.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:029.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:029.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089029378 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:29:029.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:029.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:029.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089029407 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:29:029.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:030.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:030.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089029958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:030.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:030.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:030.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089030001 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:29:030.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:030.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:29:037.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:037.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692089037776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:037.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:038.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:038.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692089038249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:038.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:038.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:038.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:038.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:038.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692089038621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:038.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:075.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:075.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692089075438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:075.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:075.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:076.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:076.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692089075615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34817 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:076.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:076.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:076.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:076.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089076624 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:076.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:123.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:123.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089123675 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:123.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:124.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:124.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692089124200 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38381 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:124.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:124.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:124.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:124.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:124.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089124727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:124.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:925.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:925.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089925724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:925.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:925.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:926.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089925933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35743 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:926.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:926.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:926.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:926.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:926.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692089926387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:926.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:926.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:926.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692089926397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:926.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:926.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:926.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089926489 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:29:926.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:926.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:926.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692089926612 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:29:926.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:926.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:926.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692089926635 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:29:926.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:927.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:927.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692089927307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:927.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:927.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:927.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692089927377 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:29:927.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:29:927.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:29:978.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:978.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692089978605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:978.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:979.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:979.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692089979089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43629 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:979.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:979.432 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:979.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:979.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:979.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692089979650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:979.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:994.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:994.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692089994756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:994.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:995.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:995.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692089995323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:995.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:29:995.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:29:995.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:29:995.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:29:995.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692089995760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:29:995.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:30:928.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:30:929.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692090928822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:30:929.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:30:929.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:30:929.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692090928977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37283 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:30:929.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:30:929.250 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:30:929.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:30:929.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:30:929.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692090929363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:30:929.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:005.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:006.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692091005826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:006.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:006.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:006.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692091006034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:006.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:006.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:31:006.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:31:006.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:006.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692091006434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:006.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:014.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:014.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692091014880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:015.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:015.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:015.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692091015074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43633 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:015.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:015.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:31:015.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:31:015.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:015.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692091015561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:015.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:922.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:922.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692091921917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:922.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:922.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:922.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692091922083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:922.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:31:922.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:31:922.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:31:922.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:31:922.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692091922504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:31:922.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:33:076.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:33:076.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692093076659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:33:076.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:33:077.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:33:077.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692093077117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35799 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:33:077.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:33:077.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:33:077.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:33:077.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:33:077.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692093077539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:33:077.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:759.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:759.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692094759647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:759.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:760.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:760.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692094760516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36843 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:760.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:760.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:34:760.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:34:760.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:760.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094760909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:760.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:760.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:760.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094760912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:760.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:761.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:761.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094761040 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:34:761.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:34:761.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:761.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094761083 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:34:761.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:34:761.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:761.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094761524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:761.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:761.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:761.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094761555 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:34:761.701 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:34:761.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:34:812.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:812.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692094812688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:812.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:813.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:813.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692094813562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:813.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:813.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:34:813.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:34:813.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:813.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692094813909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:813.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:925.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:925.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692094925502 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:925.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:926.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:926.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692094926355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:926.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:926.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:34:926.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:34:927.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:927.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094926699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:927.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:927.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:927.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094926703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:927.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:927.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:927.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094926784 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:34:927.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:34:927.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:927.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094926828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:34:927.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:34:927.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:927.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094927302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:927.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:927.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:927.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692094927339 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:34:927.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:34:927.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:34:973.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:973.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692094973484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:973.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:974.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:974.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692094974534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:974.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:34:974.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:34:974.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:34:975.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:34:975.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692094974966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:34:975.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:558.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:558.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692095558623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:558.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:559.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:559.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692095559888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:559.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:560.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:35:560.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:35:561.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095560600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:561.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:561.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095560607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:561.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:561.471 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095560703 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:35:561.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:35:561.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095560792 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:35:561.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:35:561.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095560810 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:35:561.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:35:561.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095561349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:561.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:561.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:561.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095561388 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:35:561.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:35:561.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:35:612.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:612.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692095612762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:612.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:613.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:613.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692095613534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39991 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:613.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:613.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:35:613.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:35:613.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:613.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692095613879 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:613.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:732.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:732.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692095732050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:732.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:733.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:733.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692095733049 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32889 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:733.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:733.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:35:733.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:35:733.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:733.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692095733421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:733.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:983.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:983.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692095983113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:983.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:984.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:984.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692095984050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43453 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:984.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:35:984.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:35:984.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:35:984.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:35:984.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692095984466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:35:984.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:559.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:559.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692096559000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:559.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:559.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:559.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692096559810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40183 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:559.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:560.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:36:560.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:36:560.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:560.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096560303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:560.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:749.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:749.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096749670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:749.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:750.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:750.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692096750132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:750.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:750.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:36:750.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:36:750.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:750.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096750465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:750.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:750.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:750.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096750471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:750.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:750.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:750.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096750527 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:36:750.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:36:751.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:751.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692096750758 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:36:751.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:36:751.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:751.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692096750779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:36:751.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:36:751.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:751.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692096751260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:751.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:751.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:751.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692096751286 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:36:751.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:36:751.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:36:803.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:803.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096803429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:803.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:803.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:803.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692096803857 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44141 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:803.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:36:804.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:36:804.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:36:804.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:36:804.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692096804225 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:36:804.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:402.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:403.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692097402925 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [ [ [ name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ] 2025-09-01 Mon 02:01:37:403.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVirtualNetworkAgentTrace 2025-09-01 Mon 02:01:37:404.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:405.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692097404254 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 interface_list= [ [ ] ] error_intf_list= [ [ ] ] no_config_intf_list= [ [ (*_iter151) = 00000000-0000-0000-0000-000000000000, ] ] vmi_count= [ active = 1 ] down_interface_count = 1 ] 2025-09-01 Mon 02:01:37:405.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:01:37:660.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:660.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692097660000 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 341 dequeues = 341 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 357 dequeues = 357 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692082675872 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692097404266 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:01:37:660.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:37:660.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:660.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692097660131 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 76 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 249 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 70 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 341, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:37:660.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:37:666.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:666.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692097666745 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] self_ip_list= [ [ (*_iter146) = 10.20.0.192, ] ] build_info = {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} phy_if= [ [ [ name = ens6 mac_address = fa:16:3e:63:a3:94 ], ] ] vhost_if= [ name = vhost0 mac_address = fa:16:3e:63:a3:94 ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_DEBUG log_category = sandesh_http_port = 8085 dns_servers= [ [ (*_iter149) = 10.20.0.25, (*_iter149) = 10.20.0.193, ] ] vhost_cfg= [ name = vhost0 ip = 10.20.0.192 ip_prefix_len = 24 gateway = 0.0.0.0 ] eth_name= [ [ (*_iter150) = ens6, ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.20.0.192 dns_server_list_cfg= [ [ (*_iter152) = 10.20.0.25:53, (*_iter152) = 10.20.0.193:53, (*_iter152) = 10.20.0.129:53, ] ] control_node_list_cfg= [ [ (*_iter153) = 10.20.0.25:5269, (*_iter153) = 10.20.0.193:5269, (*_iter153) = 10.20.0.129:5269, ] ] hostname_cfg = an-jenkins-deploy-platform-ansible-os-3995-2 flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 gateway_cfg_list= [ [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [ [ (*_iter155) = 10.0.0.249:8086, (*_iter155) = 10.0.0.38:8086, (*_iter155) = 10.0.0.131:8086, ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [ max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-09-01 00:11:15.602301", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-10717-1.novalocal"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name = vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [ [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [ [ (*_iter161) = 0.0.0.0, ] ] ] 2025-09-01 Mon 02:01:37:666.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:01:37:667.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:667.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692097666830 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 101 in_bytes = 7171 out_tpkts = 99 out_bytes = 5362 exception_packets = 18 exception_packets_dropped = 8 exception_packets_allowed = 10 total_flows = 0 aged_flows = 0 uptime = 1756692067401070 phy_if_info= [ [ _iter254->first = ens6 [ speed = -1 duplexity = 255 ], ] ] raw_vhost_stats= [ name = vhost0 in_pkts = 101 in_bytes = 7171 out_pkts = 99 out_bytes = 5362 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 85 in_bytes = 4774 out_pkts = 103 out_bytes = 7223 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:01:37:667.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:01:37:667.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:667.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692097666858 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:01:37:667.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:01:37:713.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:713.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692097713334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:37:713.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:713.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:713.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692097713469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36217 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:37:713.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:713.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:37:713.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:37:713.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:713.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692097713872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:37:713.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:857.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:857.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692097857505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:37:857.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:858.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:858.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692097858421 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41507 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:37:858.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:858.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:37:858.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:37:858.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:37:858.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692097858782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:37:858.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:090.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:090.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692098090598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:38:090.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:091.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:091.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692098091522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41873 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:38:091.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:091.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:38:091.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:38:091.961 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:091.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692098091906 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:38:091.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:378.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:378.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692098378462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:38:378.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:379.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:379.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692098379583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:38:379.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:379.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:38:380.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:38:380.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:380.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692098380067 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:38:380.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:38:675.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:38:676.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:01:38:676.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:38:676.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692098676011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:38:676.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:38:676.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:38:676.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692098676806 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 71 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 358, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:38:677.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:38:677.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:38:677.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:38:677.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:38:677.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:38:677.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:38:677.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Idle 2025-09-01 Mon 02:01:38:677.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:677.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692098677181 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 360, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:38:677.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:38:677.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:677.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692098677208 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:38:677.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:38:677.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:38:677.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:38:677.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692098677289 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 261 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 360, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:38:677.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:39:532.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:39:532.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692099532085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:39:532.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:39:532.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:39:532.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692099532600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35425 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:39:532.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:39:532.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:39:533.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:39:533.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:39:533.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692099533060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:39:533.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:927.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:927.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692101926915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:927.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:928.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:928.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692101928155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39409 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:928.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:928.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:41:928.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:41:928.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:928.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692101928557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:928.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:928.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:928.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692101928562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:928.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:929.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:929.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692101928717 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:41:929.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:41:929.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:929.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692101928776 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:41:929.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:41:929.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:929.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692101929355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:929.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:929.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:929.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692101929385 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:41:929.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:41:929.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:41:978.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:978.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692101978504 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:978.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:979.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:979.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692101979228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:979.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:41:979.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:41:979.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:41:979.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:41:979.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692101979584 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:41:979.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:016.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:016.276 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692102016078 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:016.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:016.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:016.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692102016815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33395 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:016.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:017.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:42:017.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:42:018.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:42:018.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:018.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102017283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:018.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:018.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:018.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102017288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:018.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:018.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:018.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102017340 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:42:018.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:018.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:018.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102017387 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:42:018.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:018.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:018.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102017960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:018.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:018.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:018.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102018035 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:42:018.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:052.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:052.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692102052198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:052.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:052.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:052.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102052756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:052.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:053.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:42:053.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:42:053.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:053.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692102053167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:053.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:053.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:053.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692102053175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:053.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:053.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:053.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692102053226 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:42:053.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:054.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:054.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692102053427 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:42:054.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:054.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:054.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692102053445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:42:054.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:054.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:054.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692102053966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:054.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:054.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:054.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692102054002 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:42:054.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:42:054.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:42:068.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:068.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692102068121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:068.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:068.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:068.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692102068731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:068.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:069.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:42:069.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:42:069.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:069.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692102069137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:069.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:106.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:106.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692102106346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:106.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:106.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:106.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692102106908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:106.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:42:107.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:42:107.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:42:107.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:42:107.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692102107250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:42:107.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:050.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:050.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692103050701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:050.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:051.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:051.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692103051612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:051.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:051.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:43:051.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:43:052.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:052.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692103051967 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:052.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:086.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:086.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692103086260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:087.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:43:087.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:087.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:088.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692103087152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:088.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:088.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:43:088.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:088.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692103088169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:088.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:160.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:160.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692103160341 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:160.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:160.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:160.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692103160638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40615 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:160.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:160.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:43:161.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:43:161.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:161.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692103161104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:161.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:673.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:673.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692103673150 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:673.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:673.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:673.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692103673618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42797 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:673.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:673.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:43:674.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:43:674.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692103674070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:674.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:674.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692103674074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:674.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:674.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692103674121 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:43:674.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:43:674.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692103674194 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:43:674.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:43:674.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692103674209 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:43:674.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:43:674.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692103674701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:674.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:674.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:674.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692103674750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:43:674.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:43:675.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:43:720.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:720.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692103720196 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:720.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:720.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:720.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692103720378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:720.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:43:720.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:43:720.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:43:720.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:720.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692103720812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:43:720.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:657.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:657.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692104656933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:657.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:657.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:657.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692104657434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46693 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:657.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:657.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:44:657.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:44:657.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:657.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692104657815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:657.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:942.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:942.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692104942323 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:942.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:943.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:943.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692104943079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45917 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:943.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:943.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:44:943.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:44:943.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:943.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692104943446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:943.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:978.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:978.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692104978119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:978.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:978.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:978.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692104978710 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:978.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:44:979.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:44:979.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:44:979.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:979.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692104979204 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:44:979.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:45:285.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:285.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692105285248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:285.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:45:286.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:286.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692105286285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40505 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:286.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:45:286.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:45:286.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:45:286.640 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:286.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692105286595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:286.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:46:584.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:46:584.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692106583930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:46:584.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:46:584.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:46:584.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692106584641 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42923 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:46:584.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:46:584.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:46:585.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:46:585.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:46:585.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692106585129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:46:585.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:903.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:903.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108903581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:903.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:904.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:904.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692108904610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35201 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:904.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:904.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:48:904.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:48:904.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:905.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108904938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:905.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:905.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:905.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108904942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:905.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:905.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:905.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108905116 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:48:905.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:48:905.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:905.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108905182 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:48:905.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:48:905.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:905.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108905718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:905.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:905.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:905.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692108905817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:48:905.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:48:906.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:48:958.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:958.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692108958170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:958.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:959.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:959.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692108959133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:959.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:48:959.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:48:959.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:48:959.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:48:959.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692108959736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:48:959.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:011.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:011.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692109011320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:011.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:011.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:011.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692109011834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:011.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:012.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:49:012.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:49:012.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:012.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:49:012.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109012191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:012.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:012.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:012.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109012198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:012.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:012.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:012.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109012250 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:49:012.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:012.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:012.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109012294 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:49:012.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:012.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:012.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109012766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:012.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:012.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:013.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109012804 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:49:013.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:062.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:062.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692109061963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:062.149 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:062.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:062.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692109062148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:062.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:062.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:49:062.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:49:062.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:062.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692109062583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:062.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:462.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109462771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:463.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:463.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692109462950 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39071 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:463.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:463.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:49:463.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:49:463.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692109463374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:463.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:463.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692109463379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:463.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:463.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692109463460 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:49:463.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:463.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692109463770 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:49:463.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:463.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:463.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692109463797 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:49:463.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:464.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:464.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692109464365 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:464.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:464.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:464.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692109464405 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:49:464.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:49:464.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:49:509.595 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:509.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692109509569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:509.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:509.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:509.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692109509895 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36593 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:509.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:510.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:49:510.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:49:510.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:510.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692109510538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:510.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:904.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:905.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692109904939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:905.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:905.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:905.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692109905410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:905.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:905.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:49:905.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:49:906.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:906.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692109906035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:906.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:998.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:998.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692109998746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:998.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:999.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:999.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692109999332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42989 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:999.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:999.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:49:999.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:49:999.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:49:999.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692109999743 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:49:999.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:293.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:293.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110293298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:293.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:293.520 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:293.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110293469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38941 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:293.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:293.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:50:293.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:50:293.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:293.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110293860 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:293.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:293.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:293.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110293868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:293.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:293.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:293.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110293942 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:50:293.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:50:294.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:294.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110294019 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:50:294.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:50:294.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:294.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692110294068 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:50:294.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:50:294.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:294.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692110294703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:294.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:294.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:294.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692110294755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:50:294.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:50:294.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:50:341.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:341.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692110341044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:341.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:341.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:341.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692110341211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:341.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:341.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:50:341.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:50:341.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:341.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692110341611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:341.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:473.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:473.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692110473709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:473.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:474.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:474.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692110474128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33559 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:474.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:50:474.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:50:474.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:50:474.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:50:474.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692110474684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:50:474.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:277.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:277.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692111277754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:277.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:277.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:277.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692111277922 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37101 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:277.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:278.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:51:278.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:51:278.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:278.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692111278320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:278.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:724.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:724.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692111724181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:724.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:724.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:724.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692111724753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36261 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:724.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:725.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:51:725.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:51:725.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:725.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692111725192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:725.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:818.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:818.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692111817883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:818.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:818.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:818.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692111818041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46187 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:818.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:51:818.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:51:818.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:51:818.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:51:818.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692111818399 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:51:818.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:52:418.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:52:419.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692112418833 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:52:419.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:52:419.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:52:419.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692112419000 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 32975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:52:419.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:52:419.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:52:419.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:52:419.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:52:419.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692112419440 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:52:419.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:53:258.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:53:258.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692113258483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:53:258.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:53:258.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:53:258.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692113258654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43737 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:53:258.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:53:258.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:53:259.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:53:259.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:53:259.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692113259045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:53:259.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:418.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:418.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692115418527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:418.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:418.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:418.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692115418725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44055 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:418.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:419.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:55:419.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:55:419.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:419.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692115419173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:419.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:419.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:419.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692115419177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:419.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:419.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:419.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692115419243 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:55:419.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:55:420.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:420.017 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692115419287 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:55:420.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:55:420.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:420.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692115419804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:420.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:420.066 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:420.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692115419847 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:55:420.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:55:419.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:55:465.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:466.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692115465942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:466.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:466.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:466.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692115466096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35721 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:466.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:466.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:55:466.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:55:466.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:466.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692115466604 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:466.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:649.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:649.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692115649320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:649.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:650.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:650.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692115650197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36489 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:650.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:650.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:55:650.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:55:650.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:650.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115650672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:650.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:650.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:650.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115650676 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:650.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:651.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:651.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115650810 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:55:651.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:55:651.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:651.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115650854 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:55:651.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:55:651.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:651.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115651344 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:651.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:651.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:651.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115651380 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:55:651.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:55:651.542 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:55:698.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:698.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692115698644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:698.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:699.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:699.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692115699101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37113 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:699.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:55:699.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:55:699.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:55:699.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:55:699.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692115699509 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:55:699.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:474.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:474.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692116474701 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:474.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:475.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:475.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692116474937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:475.037 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:475.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:56:475.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:56:475.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:475.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692116475412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:475.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:523.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:523.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692116523585 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:523.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:524.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:524.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692116524153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44443 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:524.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:524.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:56:524.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:56:524.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:524.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692116524572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:524.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:524.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:524.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692116524579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:524.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:524.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:524.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692116524651 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:56:524.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:56:524.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:524.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692116524758 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:56:524.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:56:524.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:524.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692116524797 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:56:524.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:56:525.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:525.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692116525482 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:525.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:525.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:525.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692116525660 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:56:525.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:56:525.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:56:572.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:572.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692116572899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:573.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:573.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:573.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692116573379 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40363 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:573.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:573.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:56:573.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:56:573.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:573.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692116573927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:574.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:617.630 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:617.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692116617589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:617.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:618.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:618.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692116618169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33585 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:618.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:56:618.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:56:618.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:56:618.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:56:618.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692116618737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:56:618.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:327.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:327.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692117327333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:327.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:327.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:327.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692117327729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:327.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:328.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:57:328.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:57:328.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:328.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692117328213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:328.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:328.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:328.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692117328218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:328.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:328.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:328.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692117328266 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:57:328.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:57:328.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:328.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692117328347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:57:328.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:57:328.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:328.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692117328367 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:01:57:328.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:57:328.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:328.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692117328891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:328.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:328.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:329.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692117328926 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:57:329.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:57:329.097 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:01:57:378.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:378.283 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692117378184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:378.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:378.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:378.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692117378533 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40029 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:378.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:378.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:57:378.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:57:378.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:378.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692117378910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:378.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:492.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:492.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692117492091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:492.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:492.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:492.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692117492250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:492.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:57:492.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:57:492.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:57:492.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:57:492.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692117492654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:57:492.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:315.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:315.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692118315062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:315.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:316.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:316.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692118316035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:316.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:316.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:58:316.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:58:316.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:316.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692118316478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:316.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:418.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:419.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692118418813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:419.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:419.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:419.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692118419826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:419.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:420.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:58:420.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:58:420.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:420.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692118420238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:420.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:581.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:581.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692118581499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:581.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:582.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:582.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692118581995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:582.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:58:582.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:58:582.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:58:582.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:58:582.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692118582415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:58:582.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:59:563.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:563.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:59:563.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692119562873 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:563.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:59:563.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:563.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692119563095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42601 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:563.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:59:563.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:59:563.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:563.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692119563774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:563.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:00:368.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:00:368.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692120368660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:00:368.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:00:369.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:00:369.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692120369436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35109 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:00:369.501 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:00:369.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:00:369.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:00:369.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:00:370.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692120369973 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:00:370.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:542.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:542.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692122542539 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:542.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:543.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:543.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692122543428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43703 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:543.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:543.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:543.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:544.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:544.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:02:544.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692122543777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:544.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:544.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:544.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692122543781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:544.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:544.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:544.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692122543884 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:02:544.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:544.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:544.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692122543930 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:02:544.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:544.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:544.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692122544534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:544.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:544.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:544.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692122544570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:02:544.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:596.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:596.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692122596699 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:596.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:597.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:597.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692122597113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44853 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:597.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:597.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:597.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:597.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:597.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692122597574 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:597.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:668.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:668.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692122668392 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:668.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:668.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:668.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692122668847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:668.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:669.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:669.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:669.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:669.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692122669287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:669.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:669.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:669.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692122669297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:669.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:669.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:669.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692122669419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:02:669.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:669.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:669.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692122669489 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:02:669.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:670.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:670.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692122670013 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:670.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:670.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:670.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692122670076 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:02:670.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:02:670.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:02:715.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:715.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692122715326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:715.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:716.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:716.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:02:716.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:02:716.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692122715887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42455 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:716.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:716.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:02:716.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692122716274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:02:716.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:659.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:659.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692123659631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:659.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:659.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:659.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692123659782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:659.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:660.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:03:660.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:03:660.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:660.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692123660248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:660.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:739.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:740.021 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692123739928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:740.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:740.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:740.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692123740337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33139 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:740.389 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:740.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:03:740.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:03:740.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:740.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692123740817 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:740.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:740.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:740.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692123740823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:740.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:740.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:740.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692123740900 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:03:740.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:03:740.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:741.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692123740987 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:03:741.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:03:741.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:741.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692123741002 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:03:741.097 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:03:741.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:741.551 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:03:741.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692123741473 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:741.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:741.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:741.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692123741500 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:03:741.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:03:787.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:787.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692123787391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:787.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:787.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:787.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692123787571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34377 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:787.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:787.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:03:787.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:03:787.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:788.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692123787981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:788.007 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:793.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:793.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692123793618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:793.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:793.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:793.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692123793736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40451 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:793.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:03:794.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:03:794.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:03:794.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:03:794.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692123794112 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:03:794.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:366.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:366.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:366.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44545 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:366.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:366.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:366.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:366.732 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:366.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:366.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:366.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:366.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366777 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:04:366.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:04:366.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:04:366.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:04:366.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:366.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692124366875 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:04:366.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:04:367.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:367.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692124367372 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:367.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:367.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:367.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692124367406 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:04:367.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:04:367.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:04:415.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:415.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692124415848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:416.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:416.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:416.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692124416394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40127 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:416.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:416.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:416.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:416.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:416.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692124416797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:416.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:721.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:721.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692124721722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:721.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:722.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:722.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692124722393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35003 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:722.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:04:722.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:04:722.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:04:722.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:04:722.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692124722778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:04:722.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:353.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:353.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692125352940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:353.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:354.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:354.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692125354001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35511 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:354.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:354.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:05:354.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:05:354.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:354.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692125354380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:354.507 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:766.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:766.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692125766389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:766.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:767.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:767.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692125767059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34335 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:767.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:767.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:05:767.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:05:767.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:767.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692125767464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:767.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:932.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:932.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692125931318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:932.615 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:932.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:932.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692125932769 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44915 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:932.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:05:933.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:05:933.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:05:933.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:05:933.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692125933165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:05:933.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:06:810.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:06:811.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692126810915 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:06:811.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:06:811.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:06:811.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692126811311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43143 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:06:811.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:06:811.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:06:811.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:06:811.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:06:811.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692126811705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:06:811.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:07:244.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:244.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692127244536 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:07:244.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:07:245.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:245.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692127245158 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:07:245.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:07:245.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:07:245.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:07:245.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:245.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692127245555 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:07:245.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:07:660.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:660.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692127660237 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 646 dequeues = 646 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 665 dequeues = 665 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692098677258 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692127245563 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:02:07:660.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:07:660.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:660.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692127660359 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 129 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 146 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 320, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 646, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:07:660.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:07:667.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:667.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692127667005 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.129 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:02:07:667.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:02:07:667.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:667.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692127667208 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 2238743264 used = 32524 free = 0 buffers = 0 cached = 2303930040 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.26 five_min_avg = 0.245 fifteen_min_avg = 0.165 ] cpu_share = 1.62203 ] ] 2025-09-01 Mon 02:02:07:667.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:07:667.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:667.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692127667230 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.62203 used_sys_mem = 1991069780 one_min_cpuload = 0.26 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:02:07:667.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:02:07:667.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:667.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692127667253 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 192 in_bytes = 13201 out_tpkts = 207 out_bytes = 11190 exception_packets = 22 exception_packets_dropped = 9 exception_packets_allowed = 13 raw_vhost_stats= [ name = vhost0 in_pkts = 192 in_bytes = 13201 out_pkts = 207 out_bytes = 11190 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 178 in_bytes = 9972 out_pkts = 194 out_bytes = 13253 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:02:07:667.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:07:667.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:07:667.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692127667274 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.129 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:02:07:667.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:02:08:677.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:08:678.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:02:08:678.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:08:678.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692128677440 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:678.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:678.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:08:678.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:08:678.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:08:678.391 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692128678095 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 129 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 146 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 320, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 653, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:08:678.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:08:678.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:08:678.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:08:678.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:08:678.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Idle 2025-09-01 Mon 02:02:08:678.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:678.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692128678485 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 130 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 146 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 320, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 655, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:08:678.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:08:678.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:678.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692128678513 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:08:678.799 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:08:678.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:08:678.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:08:678.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692128678594 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 130 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 147 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 488 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 139 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 321, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 655, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:08:678.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:09:533.420 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:533.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692129533292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:533.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:533.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:533.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692129533759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37083 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:533.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:534.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:534.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:534.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:09:534.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:535.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129534134 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:535.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:535.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:535.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129534147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:535.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:535.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:535.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129534203 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:535.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:535.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:535.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129534254 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:09:535.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:535.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:535.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129534760 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:535.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:535.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:535.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129534802 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:535.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:584.021 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:584.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692129583903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:584.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:584.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:584.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692129584514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43537 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:584.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:585.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:585.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:585.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:585.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692129585368 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:585.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:799.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:799.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692129799615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:799.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:800.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:800.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692129800195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:800.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:800.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:800.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:800.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:801.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692129800678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:801.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:801.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:801.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692129800685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:801.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:801.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:801.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692129800747 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:801.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:801.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:801.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692129800792 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:09:801.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:801.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:801.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692129801268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:801.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:801.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:801.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692129801307 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:09:801.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:09:801.326 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:09:850.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:850.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692129850465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:850.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:850.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:850.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692129850921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:850.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:09:851.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:09:851.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:09:851.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:09:851.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692129851303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:09:851.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:656.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:656.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692130656529 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:656.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:657.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:657.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692130657203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42135 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:657.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:657.492 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:10:657.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:10:657.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:657.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130657580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:657.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:769.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:769.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130769445 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:769.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:769.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:769.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692130769781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43831 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:769.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:770.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:10:770.150 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:10:770.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:770.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692130770190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:770.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:915.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692130915844 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:916.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:916.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692130916322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46677 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:916.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:916.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:10:916.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:10:916.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130916661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:916.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:916.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130916666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:916.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:916.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130916724 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:10:916.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:10:916.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130916817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:10:916.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:10:916.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:916.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130916842 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:10:916.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:10:917.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:917.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692130917508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:917.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:917.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:917.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692130917567 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:10:917.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:10:917.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:10:967.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:968.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692130967908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:968.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:968.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:968.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692130968071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46141 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:968.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:10:968.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:10:968.487 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:10:968.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:10:968.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692130968525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:10:968.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:493.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:493.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131493669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:493.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:494.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:494.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692131494020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43133 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:494.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:494.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:11:494.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:11:494.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:494.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131494486 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:494.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:494.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:494.540 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131494491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:494.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:494.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:494.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131494551 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:11:494.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:11:494.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:494.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131494622 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:11:494.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:11:494.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:494.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131494639 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:11:494.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:11:495.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:495.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692131495272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:495.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:495.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:495.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692131495341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:11:495.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:11:496.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:11:549.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:549.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131549213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:549.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:549.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:549.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131549669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37699 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:549.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:550.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:11:550.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:11:550.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:550.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692131550162 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:550.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:976.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:976.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692131976663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:976.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:977.258 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:977.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692131977232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:977.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:11:977.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:11:977.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:11:977.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:11:977.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692131977692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:11:977.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:549.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:549.436 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692132549285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:549.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:550.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:550.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692132550228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:550.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:550.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:12:550.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:12:550.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:550.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692132550561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:550.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:569.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:60390 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:12:581.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:60392 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:12:763.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:763.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692132763713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:763.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:764.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:764.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692132764295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45619 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:764.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:764.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:12:764.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:12:764.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:764.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692132764695 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:764.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:804.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:804.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692132804318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:804.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:804.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:805.013 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692132804966 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:805.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:12:805.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:12:805.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:12:805.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:12:805.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692132805426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:12:805.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:885.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:885.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692133885800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:885.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:886.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:886.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692133886396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42067 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:886.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:886.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:13:886.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:13:886.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:886.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692133886792 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:886.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:14:548.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:14:548.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692134548693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:14:548.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:14:549.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:14:549.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692134549239 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37715 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:14:549.269 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:14:549.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:14:549.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:14:549.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:14:549.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692134549672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:14:549.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:400.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:401.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136400871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:401.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:401.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:401.261 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136401076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:401.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:401.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:401.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:401.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:402.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136401511 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:402.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:402.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:402.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136401516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:402.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:402.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:402.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136401582 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:16:402.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:402.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:402.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:16:402.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136401645 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:16:402.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:402.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:402.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136402334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:402.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:402.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:402.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136402383 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:16:402.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:448.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:448.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692136448554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:448.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:448.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:448.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692136448871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39217 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:448.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:449.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:449.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:449.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:449.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692136449507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:449.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:909.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:909.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692136909566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:909.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:910.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:910.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692136910142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39457 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:910.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:910.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:910.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:910.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:911.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692136910564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:911.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:911.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:911.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692136910571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:911.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:911.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:911.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692136910630 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:16:911.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:911.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:911.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692136910676 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:16:911.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:911.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:911.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692136911152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:911.442 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:911.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:911.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692136911188 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:16:911.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:16:911.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:16:963.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:963.423 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692136963325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:963.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:963.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:963.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692136963887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:963.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:16:964.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:16:964.636 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:16:964.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:16:964.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692136964661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:16:964.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:484.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:484.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692137484713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:484.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:484.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:484.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692137484904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36791 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:484.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:485.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:17:485.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:17:485.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:485.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692137485349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:485.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:919.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:919.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692137918929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:919.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:919.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:919.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692137919560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33361 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:919.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:919.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:17:920.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:17:920.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:920.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692137920119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:920.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:920.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:920.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692137920129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:920.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:920.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:920.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692137920187 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:17:920.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:17:920.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:920.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692137920442 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:17:920.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:17:920.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:920.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692137920469 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:17:920.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:17:921.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:921.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692137921534 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:921.616 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:921.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:921.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:17:921.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692137921598 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:17:921.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:17:972.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:972.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692137972713 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:972.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:973.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:973.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692137973208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45803 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:973.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:17:973.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:17:973.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:17:973.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:17:974.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692137973972 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:17:974.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:035.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:035.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692138035813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:035.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:036.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:036.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138036193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41239 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:036.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:036.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:036.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:036.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:036.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692138036636 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:036.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:581.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:581.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692138581799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:581.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:582.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:582.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692138582315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 32851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:582.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:582.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:582.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:582.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:582.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138582823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:582.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:582.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:582.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138582828 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:582.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:582.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:582.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138582876 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:18:582.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:18:582.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:582.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138582948 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:18:583.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:18:583.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:583.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138582965 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:18:583.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:18:583.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:583.580 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692138583495 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:583.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:583.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:583.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692138583537 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:18:583.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:18:583.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:18:633.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:633.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692138633693 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:633.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:634.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:634.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692138634237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37013 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:634.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:634.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:634.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:634.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:634.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692138634645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:634.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:946.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:946.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692138946101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:946.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:946.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:946.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692138946610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38707 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:946.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:18:946.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:18:947.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:18:947.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:18:947.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692138947217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:18:947.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:465.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:465.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692139465513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:465.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:466.017 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:466.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692139466003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41659 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:466.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:466.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:19:466.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:19:466.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:466.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692139466531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:466.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:651.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:651.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692139651774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:651.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:652.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:652.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692139652282 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34097 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:652.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:652.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:19:652.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:19:652.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:652.913 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692139652842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:652.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:980.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:980.983 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692139980793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:980.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:981.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:982.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692139981927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39375 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:982.023 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:19:982.249 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:19:982.317 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:19:982.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:19:982.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692139982352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:19:982.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:20:801.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:20:801.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692140801286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:20:801.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:20:801.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:20:801.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692140801726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38181 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:20:801.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:20:802.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:20:802.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:20:802.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:20:802.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692140802187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:20:802.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:579.093 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:579.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692141578980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:579.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:579.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:579.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692141579457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:579.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:21:579.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:21:579.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:21:579.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:21:580.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692141579955 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:21:580.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:282.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:282.819 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143282671 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:282.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:283.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:283.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692143283187 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40747 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:283.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:283.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:283.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:283.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:283.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143283573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:283.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:283.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:283.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143283578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:283.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:283.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:283.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143283624 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:283.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:283.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:284.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143283671 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:23:284.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:284.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:284.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143284167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:284.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:284.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:284.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692143284209 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:23:284.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:23:284.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:23:332.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:332.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692143332514 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:332.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:333.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:333.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692143333020 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39097 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:333.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:23:333.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:23:333.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:23:333.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:23:333.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692143333462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:23:333.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:266.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:266.711 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692144266507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:266.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:267.164 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:267.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692144267144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41775 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:267.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:267.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:267.527 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:267.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:268.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:24:268.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144267591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:268.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:268.757 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:268.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144267603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:268.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:268.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:268.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144267691 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:24:268.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:268.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:268.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144267749 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:24:268.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:268.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:268.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144268419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:268.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:268.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:268.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144268473 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:24:268.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:317.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:317.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692144317594 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:317.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:318.042 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:318.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692144318018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42595 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:318.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:318.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:318.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:318.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:318.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692144318734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:318.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:359.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:359.741 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692144359616 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:359.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:360.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:360.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692144360343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43225 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:360.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:360.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:360.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:360.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:360.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692144360813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:360.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:582.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:582.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692144582328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:582.507 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:582.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:582.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692144582921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35469 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:582.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:583.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:583.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:583.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:583.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144583553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:583.599 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:583.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:583.646 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144583557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:583.652 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:583.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:583.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692144583654 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:24:583.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:583.740 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:583.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692144583734 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:24:583.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:583.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:583.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692144583752 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:24:583.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:584.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:584.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692144584307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:584.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:584.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:584.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692144584347 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:24:584.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:24:584.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:24:637.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:637.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692144637564 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:637.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:637.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:637.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692144637965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34725 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:637.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:24:638.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:24:638.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:24:638.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:24:638.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692144638498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:24:638.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:362.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:363.018 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692145362917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:363.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:363.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:363.169 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692145363124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:363.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:363.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:363.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:363.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:363.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692145363723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:363.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:601.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:601.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692145601653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:601.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:601.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:601.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692145601817 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45047 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:601.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:602.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:602.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:602.276 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:602.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692145602267 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:602.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:828.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:828.529 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692145828230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:828.547 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:828.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:828.567 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692145828384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45741 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:828.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:828.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:828.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:828.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:828.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692145828914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:828.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:829.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:829.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692145828924 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:829.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:829.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:829.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692145828985 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:25:829.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:829.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:829.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692145829203 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:25:830.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:830.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:830.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692145829220 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:25:830.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:830.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:830.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692145829854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:830.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:830.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:830.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692145829887 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:25:830.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:830.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:25:881.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:881.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692145881040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:881.145 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:881.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:881.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692145881485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:881.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:881.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:882.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:882.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:882.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692145882081 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:882.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:304.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:305.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692146304944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:305.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:305.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:305.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692146305131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:305.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:305.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:26:305.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:305.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:305.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692146305876 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:305.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:890.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:890.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692146890248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:890.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:890.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:891.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692146890948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43285 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:891.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:891.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:26:891.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:891.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:891.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692146891374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:891.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:27:398.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:27:398.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692147397888 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:27:398.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:27:398.115 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:27:398.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692147398068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44181 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:27:398.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:27:398.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:27:398.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:27:398.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:27:398.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692147398477 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:27:398.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:27:690.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:27:690.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692147690433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:27:690.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:27:690.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:27:690.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692147690586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:27:690.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:27:690.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:27:690.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:27:691.021 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:27:691.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692147691013 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:27:691.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:853.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:853.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692148853498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:853.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:853.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:853.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692148853884 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46875 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:853.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:854.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:28:854.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:28:854.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:854.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692148854310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:854.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:050.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:050.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692150050035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:050.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:050.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:050.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692150050647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41205 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:050.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:050.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:051.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:051.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:051.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692150051083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:051.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:051.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:051.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692150051098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:051.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:051.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:051.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692150051196 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:30:051.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:051.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:051.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692150051252 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:30:051.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:051.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:30:051.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:051.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692150051737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:051.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:051.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:051.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692150051772 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:30:051.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:096.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:097.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692150096889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:097.171 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:097.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:097.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692150097076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:097.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:097.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:097.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:097.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:097.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692150097418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:097.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:998.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:998.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692150998652 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:998.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:998.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:998.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692150998812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:998.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:999.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:30:999.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:30:999.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:999.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692150999353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:999.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:999.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:999.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692150999374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:999.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:30:999.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:999.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692150999432 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:30:999.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:999.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:999.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692150999479 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:30:999.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:30:999.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:30:999.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692150999908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:30:999.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:000.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:000.019 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692150999936 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:31:000.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:31:000.595 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:31:052.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:053.030 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151052836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:053.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:053.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:053.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151052991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46337 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:053.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:053.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:31:053.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:31:053.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:053.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692151053449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:053.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:150.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:150.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692151150579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:150.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:150.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:150.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692151150731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42153 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:150.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:150.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:31:151.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:31:151.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:151.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151151096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:151.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:759.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:759.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692151759131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:759.226 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:759.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:759.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151759328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45517 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:759.355 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:759.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:31:759.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:31:759.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:759.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151759711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:759.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:759.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:759.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151759721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:759.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:759.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:759.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692151759781 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:31:759.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:31:760.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:760.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692151760016 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:31:760.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:31:760.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:760.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692151760050 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:31:760.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:31:760.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:760.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692151760800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:760.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:760.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:760.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692151760840 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:31:760.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:31:760.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:31:807.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:807.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692151807054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:807.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:807.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:807.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692151807261 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:807.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:807.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:31:807.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:31:807.756 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:807.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692151807722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:807.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:106.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:106.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692152106623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:106.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:106.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:106.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692152106788 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46279 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:106.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:107.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:107.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:107.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:107.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692152107212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:107.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:735.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:735.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692152734902 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:735.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:735.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:735.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692152735129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:735.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:735.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:735.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:735.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:735.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692152735573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:735.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:951.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:951.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692152951240 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:951.351 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:951.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:951.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692152951402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35407 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:951.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:951.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:951.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:951.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:951.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692152951796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:951.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:138.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:138.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692153138449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:138.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:138.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:138.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692153138599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:138.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:138.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:33:139.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:33:139.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692153139203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:139.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:139.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692153139215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:139.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:139.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692153139263 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:33:139.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:33:139.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692153139341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:33:139.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:33:139.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692153139359 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:33:139.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:33:139.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692153139830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:139.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:139.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:139.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692153139875 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:33:139.908 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:33:140.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:33:187.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:187.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692153187275 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:187.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:187.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:187.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692153187880 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44983 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:187.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:188.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:33:188.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:33:188.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:188.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692153188284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:188.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:160.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:160.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692154160354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:160.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:160.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:160.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692154160546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40579 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:160.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:160.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:34:160.829 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:34:160.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:160.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692154160855 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:160.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:177.426 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:177.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692154177332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:177.495 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:178.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:178.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692154178353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38805 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:178.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:178.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:34:178.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:34:178.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:178.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692154178728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:178.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:661.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:661.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692154661653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:661.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:662.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:662.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692154662557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:662.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:34:662.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:34:662.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:34:663.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:34:663.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692154663002 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:34:663.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:249.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:249.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692156249249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:249.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:250.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:250.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156249995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41577 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:250.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:250.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:250.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:250.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:250.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692156250417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:250.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:552.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:553.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692156552264 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:553.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:555.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:555.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:556.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:557.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:36:557.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156555538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35015 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:557.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:557.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:557.857 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156556541 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:557.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:557.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:557.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156556549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:557.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:557.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:557.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156556631 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:557.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:557.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:557.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156556685 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:36:557.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:557.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:557.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156557122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:557.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:557.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:558.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692156557144 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:36:558.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:36:603.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:603.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692156603265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:603.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:603.493 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:603.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692156603401 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37615 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:603.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:36:603.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:36:603.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:36:603.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:36:603.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692156603859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:36:603.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:630.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:630.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692157630021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:630.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:630.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:630.608 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692157630573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:630.617 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:630.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:37:631.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:37:631.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:631.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692157631332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:37:631.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:37:660.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:661.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692157660786 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 952 dequeues = 952 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 974 dequeues = 974 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692128678562 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692157631339 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:02:37:661.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:37:661.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:661.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692157660948 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 190 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 216 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 49 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 723 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 206 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 474, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 952, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:37:661.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:37:667.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:667.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692157667401 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.193, ] ] ] 2025-09-01 Mon 02:02:37:667.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:02:37:667.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:667.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692157667435 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 289 in_bytes = 30087 out_tpkts = 321 out_bytes = 18116 exception_packets = 23 exception_packets_dropped = 10 raw_vhost_stats= [ name = vhost0 in_pkts = 289 in_bytes = 30087 out_pkts = 321 out_bytes = 18116 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] total_in_bandwidth_utilization = 8.30499e-15 total_out_bandwidth_utilization = 1.65612e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1532, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3055, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 277 in_bytes = 16268 out_pkts = 291 out_bytes = 30139 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:02:37:667.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:37:667.667 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:37:667.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692157667457 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:02:37:667.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:02:38:102.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:102.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692158102867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:102.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:103.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:103.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692158103294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:103.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:103.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:38:103.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:38:103.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:103.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692158103827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:103.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:103.917 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:103.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692158103834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:103.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:103.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:103.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692158103891 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:103.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:103.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:104.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692158103931 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:38:104.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:104.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:104.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:38:104.602 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692158104526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:104.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:104.640 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:104.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692158104565 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:104.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:152.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:152.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158152682 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:152.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:153.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:153.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692158153311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44079 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:153.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:153.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:38:153.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:38:153.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:153.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158153776 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:153.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:678.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:38:679.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Connect : Start Connect timer 10.0.0.131:8086 2025-09-01 Mon 02:02:38:679.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:38:679.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158678823 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.131:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:679.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:679.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:38:679.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:38:679.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:38:679.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158679420 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 193 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 218 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 731 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 208 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 479, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 968, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:38:679.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:38:679.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:38:679.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:38:679.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:38:679.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Idle 2025-09-01 Mon 02:02:38:679.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:680.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158679831 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 194 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 218 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 731 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 208 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 479, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 970, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:38:680.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:38:680.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:680.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158679857 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:680.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:680.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:38:680.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:680.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158679938 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 194 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 219 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 731 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 208 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 480, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 970, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:38:680.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:38:947.264 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:947.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692158947132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:947.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:947.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:947.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692158947547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35453 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:947.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:947.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:38:947.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:38:947.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:948.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692158947981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:948.158 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:948.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:948.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692158947987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:948.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:948.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:948.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692158948053 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:948.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:948.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:949.017 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158948430 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:949.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:949.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:949.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158948446 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:38:949.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:949.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:949.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158948928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:38:949.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:949.103 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:38:949.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692158948959 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:38:949.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:38:949.154 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:39:000.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:000.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692159000291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:000.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:000.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:000.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692159000532 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40815 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:000.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:000.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:39:000.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:39:000.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:000.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692159000910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:000.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:215.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:216.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692159215923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:216.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:216.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:216.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692159216543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:216.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:216.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:39:216.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:39:217.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:217.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692159217001 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:217.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:485.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:485.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692159485505 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:485.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:485.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:485.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692159485733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33209 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:485.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:39:486.065 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:39:486.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:39:486.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:39:486.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692159486160 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:39:486.206 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:054.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:054.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160054063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:054.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:054.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:054.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692160054781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43097 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:054.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:055.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:40:055.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:055.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:055.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692160055143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:055.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:390.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:390.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692160390526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:390.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:391.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:391.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692160391389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43113 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:391.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:391.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:40:391.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:391.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:391.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692160391715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:391.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:391.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:391.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692160391719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:391.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:392.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:392.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160391871 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:40:392.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:392.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:392.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160391977 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:40:392.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:392.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:392.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160392011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:40:392.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:392.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:40:392.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:392.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160392552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:392.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:392.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:392.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160392582 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:40:392.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:440.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:440.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692160440730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:440.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:441.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:441.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692160441423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:441.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:441.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:40:441.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:441.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:441.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692160441944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:441.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:053.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:053.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692161053131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:053.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:053.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:053.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692161053586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:053.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:053.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:41:053.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:41:054.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:054.064 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692161054012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:054.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:396.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:396.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692161396069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:396.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:396.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:396.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692161396705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:396.750 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:397.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:41:397.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:41:397.313 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:397.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692161397301 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:397.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:42:017.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:42:017.499 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692162017330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:42:017.509 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:42:017.927 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:42:017.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692162017903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:42:017.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:42:018.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:42:018.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:42:018.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:42:018.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692162018310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:42:018.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:341.535 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:341.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692163341436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:341.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:342.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:342.377 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692163342333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36977 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:342.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:342.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:342.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:342.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:342.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692163342739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:342.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:518.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:518.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692163518320 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:518.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:519.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:519.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692163518990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45407 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:519.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:519.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:519.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:519.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:520.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163519407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:520.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:520.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:520.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163519415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:520.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:520.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:520.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163519485 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:520.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:520.417 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:520.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163519547 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:43:520.449 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:520.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:520.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163520149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:520.484 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:520.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:520.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163520198 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:43:520.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:43:520.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:43:569.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:569.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692163569346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:569.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:569.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:569.934 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692163569864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 41163 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:569.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:570.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:43:570.231 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:43:570.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:570.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692163570272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:43:570.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:605.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:605.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692164605367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:605.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:606.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:606.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692164606107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:606.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:44:606.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:44:606.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:44:606.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:606.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692164606528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:44:606.589 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:086.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:086.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692165086144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:086.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:086.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:086.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692165086603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:086.658 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:086.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:45:086.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:45:086.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:087.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692165086970 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:087.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:087.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:087.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692165086995 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:087.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:087.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:087.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692165087057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:45:087.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:45:087.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:087.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692165087101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:45:087.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:45:087.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:087.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692165087573 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:087.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:087.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:087.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692165087615 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:45:087.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:45:088.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:45:136.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:136.548 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692165136209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:136.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:136.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:136.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692165136402 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39397 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:136.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:45:136.741 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:45:136.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:45:136.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:45:136.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692165136870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:45:136.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:046.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:046.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692166046156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:046.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:047.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:047.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692166047870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45473 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:047.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:048.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:048.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:048.549 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:048.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692166048480 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:048.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:230.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:230.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692166230405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:230.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:231.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:231.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692166231433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:231.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:231.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:231.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:233.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166231883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:233.291 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:233.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166231889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:233.336 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:233.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166232083 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:46:233.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:46:233.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166232211 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:46:233.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:46:233.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166232231 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:46:233.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:46:233.498 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166233167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:233.521 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:233.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:233.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166233207 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:46:233.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:46:233.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:46:282.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:282.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692166282677 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:282.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:283.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:283.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692166283425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43935 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:283.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:283.727 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:283.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:283.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:283.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692166283787 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:283.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:658.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:658.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692166658651 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:658.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:659.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:659.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692166659243 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39569 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:659.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:46:659.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:46:660.056 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:46:660.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:46:660.125 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692166660082 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:46:660.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:014.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:015.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692167014865 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:015.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:015.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:015.789 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692167015764 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37133 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:015.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:016.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:47:016.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:47:016.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:016.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692167016206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:016.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:016.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:016.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692167016212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:016.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:016.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:016.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692167016346 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:47:016.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:47:016.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:016.660 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692167016434 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:47:016.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:47:016.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:016.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692167016448 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:47:016.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:47:017.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:017.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692167016954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:017.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:017.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:017.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692167016989 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:47:017.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:47:017.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:47:066.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:066.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692167066163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:066.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:067.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:067.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692167067170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45373 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:067.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:067.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:47:067.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:47:067.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:067.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692167067581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:067.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:211.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:211.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692167210974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:211.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:212.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:212.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692167212593 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:212.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:212.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:47:213.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:47:213.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:213.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692167213407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:213.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:967.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:967.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692167967749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:967.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:968.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:968.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692167968171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35219 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:968.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:47:968.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:47:968.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:47:968.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:47:968.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692167968667 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:47:968.737 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:48:010.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:48:010.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692168010607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:48:010.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:48:011.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:48:011.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692168011158 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37455 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:48:011.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:48:011.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:48:011.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:48:011.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:48:011.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692168011681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:48:011.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:175.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:175.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692169175591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:175.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:176.135 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:176.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692169176120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:176.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:176.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:49:176.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:49:176.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:176.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692169176600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:176.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:948.918 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:948.979 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692169948814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:948.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:949.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:949.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692169949766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39041 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:949.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:49:950.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:49:950.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:49:950.197 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:49:950.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692169950149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:49:950.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:944.312 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:944.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692170944211 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:944.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:944.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:945.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692170944919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 40739 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:945.020 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:945.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:945.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:945.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:945.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692170945272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:945.886 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:945.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:945.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692170945292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:945.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:945.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:945.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:50:945.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692170945371 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:945.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:946.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:946.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692170945434 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:50:946.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:946.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:946.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692170945892 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:946.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:946.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:946.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692170945925 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:50:946.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:50:996.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:996.174 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692170996072 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:996.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:996.653 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:996.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692170996626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46349 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:996.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:50:996.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:50:996.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:50:996.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:50:996.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692170996968 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:50:997.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:611.922 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:611.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692171611814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:611.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:612.404 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:612.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692171612385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36727 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:612.441 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:612.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:612.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:612.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:613.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:51:613.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692171612806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:613.558 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:613.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:613.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692171612815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:613.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:613.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:613.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692171612873 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:51:613.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:51:613.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:613.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692171612916 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:51:613.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:51:613.662 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:613.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692171613393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:613.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:613.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:613.695 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692171613438 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:51:613.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:51:666.626 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:666.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692171666523 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:666.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:667.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:667.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692171667006 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43953 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:667.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:667.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:667.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:667.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:667.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692171667446 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:667.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:996.223 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:996.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692171996104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:996.304 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:996.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:996.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692171996690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40445 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:996.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:51:997.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:51:997.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:51:997.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:51:997.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692171997191 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:51:997.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:693.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:693.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692172693560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:693.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:694.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:694.236 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692172694149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37931 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:694.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:52:694.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:52:694.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:52:694.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:52:694.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692172694559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:52:694.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:316.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:316.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173316737 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:316.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:317.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:317.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692173317647 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37811 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:317.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:317.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:53:317.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:53:318.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.741 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318015 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:318.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:318.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.766 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318037 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:318.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:318.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318115 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:53:318.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:318.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318200 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:53:318.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:318.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318218 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:53:318.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:318.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.900 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318669 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:318.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:318.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:318.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173318691 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:53:318.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:318.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:53:364.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:364.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692173364080 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:364.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:364.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:364.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692173364433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38563 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:364.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:364.739 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:53:364.794 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:53:364.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:364.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173364827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:364.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:851.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:851.541 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692173851380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:851.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:852.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:852.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692173852031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44151 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:852.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:852.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:53:852.342 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:53:852.407 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:852.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692173852396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:852.459 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:982.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692173982209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:982.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:982.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692173982363 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37263 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:982.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:982.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:53:982.767 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:53:982.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173982797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:982.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:982.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173982803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:982.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:982.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173982852 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:53:982.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:982.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173982917 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:53:982.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:982.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:982.985 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692173982933 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:53:982.994 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:983.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:983.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692173983489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:53:983.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:983.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:53:983.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692173983526 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:53:983.606 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:53:983.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:54:032.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:032.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692174032847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:032.964 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:033.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:033.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692174033367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45839 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:033.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:033.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:54:033.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:54:033.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:033.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692174033836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:033.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:319.286 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:319.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692174318962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:319.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:319.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:319.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692174319690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42107 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:319.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:320.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:54:320.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:54:320.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:320.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692174320120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:320.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:710.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:710.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692174710700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:710.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:711.220 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:711.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692174711174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:711.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:711.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:54:711.537 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:54:711.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:711.622 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692174711592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:711.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:55:078.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:078.163 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692175077975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:55:078.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:55:078.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:078.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692175078569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36753 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:55:078.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:55:078.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:55:078.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:55:079.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:079.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692175079009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:55:079.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:174.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:174.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692176174272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:174.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:174.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:174.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692176174755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33711 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:174.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:56:175.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:56:175.573 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:56:175.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:175.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692176175609 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:56:175.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:005.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:005.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692177005147 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:005.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:005.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:005.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692177005762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35599 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:005.808 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:006.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:57:006.531 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:57:006.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:006.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692177006559 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:006.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:136.643 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:136.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692178136610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:136.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:136.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:136.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692178136775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:136.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:137.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:137.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:137.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:137.823 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:58:137.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178137169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:137.895 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:137.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:137.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178137177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:137.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:137.926 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:137.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178137243 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:58:137.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:137.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:137.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178137287 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:58:137.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:137.997 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:138.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178137763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:138.011 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:138.017 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:138.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178137806 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:58:138.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:182.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:183.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692178182933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:183.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:183.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:183.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178183455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:183.533 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:183.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:183.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:183.911 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:183.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178183901 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:183.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:671.800 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:671.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692178671763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:671.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:672.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:672.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692178672366 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43883 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:672.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:672.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:672.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:672.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:672.763 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178672733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:672.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:672.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:672.785 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692178672740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:672.791 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:672.821 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:672.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692178672815 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:58:672.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:672.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:672.878 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692178672860 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:58:672.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:673.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:673.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692178673302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:673.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:673.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:673.444 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692178673370 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:58:673.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:58:673.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:58:722.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:722.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692178722578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:722.796 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:722.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:722.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692178722736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39017 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:722.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:58:723.040 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:58:723.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:58:723.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:58:723.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692178723142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:58:723.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:192.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:192.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692179192030 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:192.256 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:192.650 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:192.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692179192638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34499 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:192.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:192.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:193.104 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:193.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:193.228 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179193164 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:193.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:749.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:749.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179749283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:749.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:749.840 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:749.870 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692179749818 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:749.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:750.179 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:750.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:750.322 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:750.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692179750314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:750.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:811.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:811.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692179811732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:811.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:812.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:812.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692179812270 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:812.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:812.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:812.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:812.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:812.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179812692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:812.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:812.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:812.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179812700 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:812.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:812.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:812.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692179812801 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:59:812.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:59:813.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:813.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179813229 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:59:813.812 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:59:813.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:813.833 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179813246 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:02:59:813.839 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:59:813.845 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:813.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179813706 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:813.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:813.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:813.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179813728 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:59:813.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:59:813.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:02:59:867.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:867.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692179867041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:867.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:867.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:867.849 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692179867782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42039 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:867.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:868.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:868.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:868.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:868.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692179868230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:868.267 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:750.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:750.772 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692180750679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:750.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:750.865 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:750.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692180750859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46659 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:750.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:751.201 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:00:751.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:00:751.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692180751281 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:751.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:751.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.340 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692180751285 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:751.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:751.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692180751394 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:00:751.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:00:751.715 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692180751472 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:00:751.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:00:751.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692180751487 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:00:751.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:00:751.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.958 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692180751939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:751.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:751.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:751.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692180751976 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:00:752.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:00:752.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:00:804.453 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:804.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692180804417 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:804.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:804.858 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:804.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692180804850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35291 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:804.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:805.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:00:805.263 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:00:805.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:805.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692180805288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:805.353 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:894.414 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:894.474 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692180894374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:894.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:894.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:894.779 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692180894757 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42893 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:894.786 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:00:895.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:00:895.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:00:895.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:00:895.324 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692180895286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:00:895.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:155.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:155.397 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692181155303 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:155.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:155.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:155.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692181155863 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42621 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:155.944 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:156.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:01:156.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:01:156.609 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:156.629 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692181156596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:156.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:604.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:604.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692181604434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:604.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:604.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:604.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692181604622 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43277 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:604.717 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:604.953 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:01:605.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:01:605.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:605.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692181605092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:605.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:732.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:732.649 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692181732463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:732.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:733.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:733.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692181733190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:733.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:01:733.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:01:733.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:01:733.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:01:733.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692181733645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:01:733.685 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:001.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:001.632 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692183001461 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:001.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:002.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:002.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692183002683 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44393 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:002.746 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:003.084 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:03:003.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:03:003.373 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:003.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692183003310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:003.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:803.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:803.907 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692183803747 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:803.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:804.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:804.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692183804633 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33895 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:804.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:03:804.967 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:03:805.094 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:03:805.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:03:805.182 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692183805119 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:03:805.192 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:205.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:205.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692185205238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:205.465 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:205.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:205.483 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692185205410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 43549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:205.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:205.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:205.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:205.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:205.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692185205861 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:205.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:205.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:205.914 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692185205871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:205.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:205.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:205.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692185205940 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:05:205.978 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:205.987 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:206.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692185205980 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:05:206.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:206.562 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:206.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692185206526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:206.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:206.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:206.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:05:206.610 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692185206576 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:05:206.637 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:252.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:252.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692185252718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:252.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:253.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:253.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692185253335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34477 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:253.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:253.642 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:253.697 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:253.748 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:253.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692185253739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:253.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:440.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:440.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692185440772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:440.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:441.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:441.292 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692185441249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34827 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:441.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:441.620 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:441.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:441.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:441.764 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692185441727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:441.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:441.783 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:441.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692185441732 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:441.802 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:441.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:441.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692185441811 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:05:441.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:441.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:441.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692185441877 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:05:441.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:442.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:442.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692185442338 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:442.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:442.382 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:05:442.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:442.463 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692185442360 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:05:442.476 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:05:494.572 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:494.659 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692185494531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:494.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:494.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:494.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692185494685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41721 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:494.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:05:494.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:05:495.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:05:495.069 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:05:495.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692185495062 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:05:495.102 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:261.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:261.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692186261867 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:261.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:262.485 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:262.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692186262451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:262.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:262.776 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:06:262.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:06:262.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:262.956 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692186262927 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:262.972 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:494.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:494.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692186494188 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:494.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:494.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:494.601 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692186494579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34663 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:494.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:494.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:06:494.990 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:06:495.046 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:06:495.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692186495019 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:06:495.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:251.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:251.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692187251371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:251.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:251.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:251.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187251546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:251.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:251.852 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:07:251.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:07:251.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.043 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187251954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:252.051 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:252.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.067 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187251958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:252.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:252.083 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187252033 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:07:252.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:252.148 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.168 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187252108 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:07:252.176 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:252.189 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187252123 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:07:252.211 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:252.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.873 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692187252765 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:252.880 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:252.890 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:252.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692187252823 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:07:252.935 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:252.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:07:305.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:305.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692187305137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:305.234 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:305.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:305.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692187305310 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 35671 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:305.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:305.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:07:305.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:07:305.703 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:305.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692187305678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:305.729 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:661.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:661.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187661053 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 1268 dequeues = 1268 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 1293 dequeues = 1293 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692158679907 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692187305700 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:03:07:661.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:07:661.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:661.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187661184 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 255 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 286 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 966 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 275 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 631, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1268, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:07:661.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:07:667.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:667.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187667551 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.129, (*_iter149) = 10.20.0.25, ] ] ] 2025-09-01 Mon 02:03:07:667.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:03:07:667.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:667.702 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187667588 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ num_cpu = 2 sys_mem_info= [ total = 2213552864 used = 32524 free = 0 buffers = 0 cached = 2303930040 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18275337 res = 0 ] cpuload= [ one_min_avg = 0.375 five_min_avg = 0.255 fifteen_min_avg = 0.175 ] cpu_share = 1.50833 ] ] 2025-09-01 Mon 02:03:07:667.712 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:07:667.723 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:667.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187667594 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 1.50833 used_sys_mem = 1991069780 one_min_cpuload = 0.375 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:03:07:667.754 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:03:07:667.769 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:667.864 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187667616 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 380 in_bytes = 36149 out_tpkts = 414 out_bytes = 22918 exception_packets = 26 exception_packets_allowed = 16 raw_vhost_stats= [ name = vhost0 in_pkts = 380 in_bytes = 36149 out_pkts = 414 out_bytes = 22918 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 355 in_bytes = 20440 out_pkts = 382 out_bytes = 36201 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:07:667.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:07:667.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:667.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187667646 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:07:667.919 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:07:909.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:909.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187909227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:909.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:909.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:909.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692187909783 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36609 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:909.822 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:910.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:07:910.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:07:910.241 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:910.378 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692187910230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:910.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:910.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:910.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692187910237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:910.411 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:910.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:910.440 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692187910300 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:07:910.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:911.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:911.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187910510 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:07:911.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:911.430 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:911.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187910527 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:07:911.451 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:911.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:911.466 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187911260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:911.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:911.479 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:911.497 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187911310 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:07:911.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:07:911.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:07:963.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:963.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692187963462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:963.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:963.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:963.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692187963916 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:963.955 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:07:964.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:07:964.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:07:964.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:07:964.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692187964316 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:07:964.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:331.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:331.971 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692188331813 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:331.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:332.332 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:332.357 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692188332304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:332.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:332.611 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:08:332.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:08:332.724 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:332.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692188332705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:332.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:333.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:333.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692188332976 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:333.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:333.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:333.270 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692188333223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45733 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:333.277 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:333.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:08:333.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:08:333.586 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:333.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692188333578 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:333.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:457.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:457.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692188457144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:457.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:457.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:457.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692188457728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38739 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:457.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:457.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:08:458.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:08:458.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:458.204 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692188458130 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:458.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:679.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:08:680.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Connect : Start Connect timer 10.0.0.249:8086 2025-09-01 Mon 02:03:08:680.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:08:680.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692188680072 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.249:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:08:680.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:08:680.574 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:08:680.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:08:680.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:08:680.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692188680511 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.249:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 259 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 291 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 983 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 280 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 642, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1296, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:08:680.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:08:680.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:08:680.843 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:08:680.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:08:680.960 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Idle 2025-09-01 Mon 02:03:08:681.074 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:681.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692188680888 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 291 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 983 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 280 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 642, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1298, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:08:681.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:08:681.217 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:681.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692188680913 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:08:681.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:08:681.251 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:08:681.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:681.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692188681064 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 292 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 983 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 280 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 643, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1298, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:08:681.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:08:873.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:873.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692188873467 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:873.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:874.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:874.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692188874089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:874.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:08:874.394 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:08:874.478 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:08:874.561 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:08:874.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692188874527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:08:874.594 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:295.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:295.073 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692190294882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:295.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:295.385 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:295.402 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692190295367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:295.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:295.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:10:295.782 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:10:295.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:295.902 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692190295850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:295.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:908.743 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:908.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692190908638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:908.817 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:909.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:909.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692190909263 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:909.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:10:909.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:10:909.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:10:909.836 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:10:909.854 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692190909816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:10:909.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:933.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:933.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692191933723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:933.892 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:934.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:934.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692191934202 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39545 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:934.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:934.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:11:934.634 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:11:934.686 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:934.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692191934661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:934.713 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:934.721 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:934.730 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692191934668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:934.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:934.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:934.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692191934746 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:11:934.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:934.792 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:934.805 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692191934788 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:11:934.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:935.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:935.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692191935284 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:935.367 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:935.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:935.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692191935323 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:11:935.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:11:935.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:11:981.663 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:981.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692191981550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:981.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:982.060 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:982.087 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692191982035 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 45481 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:982.095 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:11:982.375 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:11:982.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:11:982.526 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:11:982.545 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692191982507 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:11:982.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:490.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:490.380 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692192490266 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:490.395 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:490.921 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:490.965 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692192490893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38657 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:490.974 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:491.194 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:12:491.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:12:491.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:491.976 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:12:492.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192491298 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:492.122 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:492.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:492.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192491307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:492.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:492.156 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:492.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192491384 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:12:492.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:12:492.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:492.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192491440 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:12:492.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:12:492.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:492.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192491919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:492.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:492.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:492.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192491959 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:12:492.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:12:538.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:538.202 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692192538108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:538.212 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:538.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:538.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692192538684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:538.774 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:12:538.992 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:12:539.044 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:12:539.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:12:539.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692192539076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:12:539.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:026.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:026.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692193026631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:026.798 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:027.229 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:027.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692193027217 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35351 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:027.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:029.034 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:13:029.131 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:13:029.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:029.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692193029162 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:029.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:387.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:46486 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:13:399.912 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.192:8085::10.20.0.193:46494 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:13:601.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:601.458 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692193601258 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:601.468 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:601.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:601.489 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692193601425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:601.495 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:601.720 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:13:601.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:13:601.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:601.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692193601841 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:601.868 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:968.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:968.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692193967985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:968.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:968.457 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:968.481 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692193968448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:968.502 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:968.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:13:968.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:13:968.945 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:968.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692193968926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:969.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:969.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:969.033 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692193968932 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:969.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:969.045 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:969.070 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692193968977 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:13:969.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:13:969.114 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:969.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692193969077 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:13:969.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:13:969.153 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:969.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692193969098 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:13:969.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:13:969.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:969.738 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692193969649 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:13:969.752 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:969.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:969.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692193969690 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:13:969.787 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:13:969.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:14:023.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:023.151 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692194022947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:023.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:023.656 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:023.683 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692194023640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44011 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:023.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:023.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:14:024.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:14:024.089 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:024.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692194024063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:024.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:978.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:978.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692194977938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:978.117 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:978.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:978.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692194978556 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 35401 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:978.621 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:978.869 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:14:978.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:14:979.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:979.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692194978994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:979.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:979.227 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:979.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692194979003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:979.253 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:979.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:979.282 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692194979096 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:14:979.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:14:979.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:979.936 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692194979351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:14:979.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:14:979.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:979.968 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692194979367 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:14:979.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:14:979.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:979.996 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692194979858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:14:980.002 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:14:980.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:980.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692194979883 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:14:980.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:14:980.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:15:031.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:031.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692195031215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:031.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:031.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:031.872 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692195031799 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46149 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:031.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:032.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:15:032.170 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:15:032.218 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:032.248 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692195032206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:032.255 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:077.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:077.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692195077210 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:077.306 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:077.384 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:077.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692195077378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40501 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:077.405 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:077.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:15:077.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:15:077.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:077.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692195077889 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:077.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:099.400 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:099.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692195099288 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:099.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:099.879 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:099.901 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692195099848 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46669 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:099.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:100.143 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:15:100.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:15:100.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:100.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692195100262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:100.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:564.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:564.129 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692195563947 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:564.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:564.568 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:564.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692195564549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45591 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:564.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:15:564.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:15:564.871 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:15:564.931 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:15:564.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692195564922 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:15:564.980 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:013.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:013.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692196013351 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:013.575 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:014.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:014.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692196014040 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:014.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:014.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:16:014.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:16:014.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:014.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692196014571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:014.596 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:986.126 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:986.180 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692196986028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:986.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:986.954 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:986.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692196986942 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:986.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:16:987.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:16:987.225 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:16:987.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:16:987.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692196987248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:16:987.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:850.810 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:850.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692197850707 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:850.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:851.645 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:851.674 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692197851621 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:851.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:17:851.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:17:851.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:17:852.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:17:852.091 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692197852031 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:17:852.098 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:988.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:988.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692198988393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:988.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:988.811 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:988.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692198988804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 33355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:988.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:989.159 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:18:989.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:18:989.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:989.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692198989287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:989.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:989.358 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:989.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692198989294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:989.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:989.387 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:989.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692198989376 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:989.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:989.446 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:989.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692198989415 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:18:989.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:18:989.861 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:989.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692198989847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:18:989.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:18:989.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:18:989.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:18:989.906 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692198989876 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:18:990.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:19:041.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:041.127 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199041048 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:041.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:041.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:041.365 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199041205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46861 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:041.371 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:041.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:19:041.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:19:041.563 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:041.583 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692199041557 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:041.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:489.243 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:489.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692199489083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:489.311 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:489.815 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:489.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692199489781 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45503 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:489.897 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:490.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:19:490.205 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:19:490.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:491.412 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199490243 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:491.431 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:491.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:491.455 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199490251 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:491.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:491.473 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:491.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199490335 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:19:491.514 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:19:491.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:491.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199490390 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:19:491.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:19:491.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:491.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199490883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:491.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:491.577 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:491.591 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199490931 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:19:491.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:19:490.949 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:19:541.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:541.334 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692199541233 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:541.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:541.835 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:541.874 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692199541809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 35825 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:541.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:19:542.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:19:542.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:19:542.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:19:542.318 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692199542287 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:19:542.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:112.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:112.938 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692200112668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:112.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:112.959 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:112.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692200112830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 38949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:112.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:113.118 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:20:113.167 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:20:113.224 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:113.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692200113215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:113.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:568.553 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:568.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692200568462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:568.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:569.196 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:569.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692200569118 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:569.245 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:20:569.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:20:569.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:20:569.556 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:20:569.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692200569545 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:20:569.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:163.505 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:163.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692201163382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:163.565 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:164.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:164.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692201163986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40795 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:164.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:164.346 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:21:164.418 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:21:164.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:164.654 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692201164506 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:164.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:164.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:164.681 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692201164515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:164.687 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:164.693 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:164.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692201164581 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:21:164.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:164.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:164.866 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692201164796 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:21:164.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:164.894 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:164.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692201164812 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:21:164.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:165.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:165.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201165330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:165.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:165.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:165.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201165367 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:21:165.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:165.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:21:211.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:211.554 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201211471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:211.564 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:211.694 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:211.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692201211665 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44855 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:211.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:211.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:21:212.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:21:212.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:212.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692201212121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:212.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:848.191 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:848.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692201848154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:848.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:848.310 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:848.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692201848304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 44459 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:848.344 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:848.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:21:848.925 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:21:848.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.005 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692201848955 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:849.012 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:849.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692201848965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:849.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:849.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692201849042 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:21:849.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:849.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.157 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201849127 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:21:849.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:849.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201849148 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:21:849.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:849.673 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.684 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201849666 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:849.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:849.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:849.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201849703 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:21:849.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:21:849.725 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:21:898.973 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:899.048 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201898834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:899.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:899.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:899.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692201899465 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 34073 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:899.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:899.846 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:21:899.928 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:21:900.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:900.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692201899981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:900.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:913.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:913.488 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692201913302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:913.496 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:913.957 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:913.984 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692201913925 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:913.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:21:914.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:21:914.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:21:914.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:21:914.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692201914293 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:21:914.345 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:238.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:238.429 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692202238241 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:238.439 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:238.809 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:238.837 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692202238778 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 34749 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:238.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:239.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:22:239.130 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:22:239.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:239.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692202239155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:239.252 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:675.760 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:675.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692202675664 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:675.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:676.982 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:22:676.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:677.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:22:677.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692202676944 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:677.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:677.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:677.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692202677135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:677.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:845.222 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:845.274 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692202845116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:845.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:845.675 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:845.699 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692202845662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 44661 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:845.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:22:846.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:22:846.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:22:846.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:22:846.183 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692202846136 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:22:846.190 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:111.472 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:111.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692204111325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:111.588 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:111.597 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:111.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692204111487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38127 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:111.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:111.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:24:111.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:24:111.862 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:111.882 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692204111854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:111.888 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:826.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:826.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692204826259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:826.364 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:826.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:826.941 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692204826874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:826.950 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:827.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:24:827.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:24:827.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:24:827.421 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692204827397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:24:827.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:090.614 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:090.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692206090468 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:090.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:090.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:090.700 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692206090631 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 46831 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:090.706 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:090.932 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:090.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:091.059 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:091.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:26:091.755 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206091050 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:091.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:091.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:091.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206091056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:091.820 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:091.826 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:091.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206091114 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:091.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:091.867 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:091.881 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206091159 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:26:091.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:091.896 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:091.905 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206091668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:091.910 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:091.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:091.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206091719 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:091.937 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:143.887 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:143.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692206143851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:143.962 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:144.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:144.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692206144295 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 36149 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:144.329 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:144.966 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:145.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:145.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:145.166 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692206145128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:145.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:709.350 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:709.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692206709322 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:709.416 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:709.733 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:709.758 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692206709725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 39687 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:709.765 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:710.079 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:710.172 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:710.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:710.259 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692206710198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:710.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:710.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:710.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692206710203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:710.307 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:710.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:710.335 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692206710255 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:710.347 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:710.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:710.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692206710298 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:26:710.374 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:710.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:710.883 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206710806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:710.891 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:710.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:710.929 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692206710849 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:710.940 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:711.077 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:26:756.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:756.284 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692206756198 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:756.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:756.388 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:756.403 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692206756381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 37249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:756.410 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:26:756.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:26:756.790 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:756.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:756.877 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692206756826 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:26:756.885 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:090.396 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:090.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692207090291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:090.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:091.390 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:091.413 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692207091377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 46227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:091.419 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:091.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:27:091.875 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:27:091.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:091.991 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692207091910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:091.998 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:784.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:784.128 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692207784012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:784.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:784.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:784.680 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692207784627 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 38313 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:784.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:785.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:27:785.099 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:27:785.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692207785128 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:785.271 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:785.279 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.287 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692207785135 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:785.293 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:785.299 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692207785186 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:27:785.338 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:27:785.409 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.437 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692207785400 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:27:785.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:27:785.456 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692207785418 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:27:785.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:27:785.920 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.933 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692207785890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:785.939 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:785.951 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:785.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692207785930 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:27:785.977 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:27:785.981 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:27:800.963 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:801.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692207800937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:801.009 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:801.298 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:801.309 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692207801292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 40479 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:801.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:801.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:27:801.638 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:27:801.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:801.744 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692207801687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:801.753 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:831.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:831.199 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692207831113 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:831.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:831.668 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:831.705 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692207831638 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41975 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:831.714 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:27:831.893 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:27:831.947 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:27:831.989 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:27:832.031 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692207831980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:27:832.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:813.165 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:813.232 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692208813129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:813.244 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:813.770 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:813.816 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692208813730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 42301 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:813.828 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:814.055 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:28:814.100 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:28:814.141 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:814.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208814125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:814.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:895.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:895.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692208895518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:895.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:896.480 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:896.528 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208896468 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 40967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:896.536 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:896.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:28:896.806 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:28:897.215 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.265 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208896831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:897.273 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:897.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208896835 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:897.296 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:897.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.323 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208896911 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:28:897.333 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:28:897.341 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.356 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208897012 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:28:897.366 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:28:897.372 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.386 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208897037 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:28:897.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:28:897.525 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.534 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208897518 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:897.539 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:897.546 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:897.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692208897541 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:28:897.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:28:897.571 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:28:949.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:949.831 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692208949672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:949.841 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:950.280 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:950.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692208950238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:950.316 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:950.559 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:28:950.625 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:28:950.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:950.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692208950678 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:950.728 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:216.161 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:216.238 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692209216110 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:216.321 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:216.641 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:216.690 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692209216628 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 43203 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:216.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:216.970 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:29:217.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:29:217.120 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:217.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692209217108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:217.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:763.942 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:763.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692209763816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:764.004 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:764.424 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:764.447 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692209764406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:764.454 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:764.679 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:29:764.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:29:764.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:764.853 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692209764804 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:764.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:877.948 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:877.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692209877795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:878.008 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:878.339 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:878.368 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692209878321 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37339 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:878.376 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:878.605 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:29:878.688 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:29:878.761 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:878.804 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692209878734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:878.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:812.434 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:812.486 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692210812244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:812.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:813.076 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:813.109 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692210813046 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34191 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:813.116 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:30:813.315 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:30:813.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:30:813.445 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:30:813.475 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692210813434 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:30:813.482 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:894.952 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:895.003 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692211894843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:895.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:895.797 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:895.824 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692211895786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 39133 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:895.832 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:31:896.106 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:31:896.142 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:31:896.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:31:896.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692211896167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:31:896.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:033.330 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:033.383 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213033230 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:033.393 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:034.111 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:034.137 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692213034100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 33189 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:034.144 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:034.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:034.433 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:035.160 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:035.185 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213034460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:035.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:035.200 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:035.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213034464 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:035.213 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:035.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:035.247 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213034564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:035.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:035.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:035.281 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213034629 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:33:035.288 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:035.294 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:035.302 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213035117 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:035.308 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:035.314 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:035.327 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213035148 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:035.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:035.362 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:33:082.543 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:082.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692213082452 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:082.603 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:083.175 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:083.195 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692213083155 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38037 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:083.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:083.461 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:083.500 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:083.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:083.581 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692213083524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:083.600 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:977.124 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:977.177 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692213976986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:977.188 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:977.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:977.515 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692213977475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 45773 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:977.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:977.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:33:977.842 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:33:977.889 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:977.903 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692213977868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:977.909 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:977.916 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:977.924 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692213977872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:977.930 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:978.181 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:978.240 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692213977969 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:978.268 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:978.278 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:978.295 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692213978013 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:33:978.301 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:978.494 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:978.506 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692213978487 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:33:978.513 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:33:978.538 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:33:978.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692213978531 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:33:978.570 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:33:978.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:34:030.016 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:030.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692214029918 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:030.092 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:030.406 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:030.427 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692214030377 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37075 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:030.435 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:030.665 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:34:030.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:34:030.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:030.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692214030786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:030.813 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:037.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:037.707 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692214037619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:037.716 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:037.781 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:037.801 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692214037774 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 46435 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:037.807 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:038.082 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:34:038.139 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:34:038.186 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:038.207 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692214038177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:038.214 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:701.775 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:701.834 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692214701592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:701.847 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:701.859 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:701.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692214701796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 36683 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:701.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:702.088 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:34:702.138 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:34:702.216 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:702.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692214702208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:702.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:702.246 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:702.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692214702214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:702.260 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:702.320 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:702.352 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692214702307 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:34:702.363 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:34:702.408 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:702.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692214702403 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:34:702.443 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:34:702.450 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:702.464 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692214702418 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:34:702.469 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:34:703.035 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:703.053 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692214702977 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:703.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.25 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:34:703.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:703.585 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:703.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692214703029 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:34:703.618 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:34:753.300 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:753.360 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692214753172 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:753.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:753.379 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:753.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692214753325 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 37465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:753.398 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:34:753.631 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:34:753.696 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:34:753.736 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:34:753.762 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692214753727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:34:753.771 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:066.105 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:066.173 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692215065962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:066.184 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:066.193 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:066.203 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692215066125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 37413 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:066.209 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:066.467 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:35:066.530 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:35:066.592 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:066.647 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692215066583 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:066.655 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:663.085 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:663.198 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692215662919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:663.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:663.219 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:663.230 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692215663108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 42641 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:663.235 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:663.415 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:35:663.490 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:35:663.557 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:663.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692215663547 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:663.584 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:712.448 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:712.511 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692215712292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:712.523 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:712.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:712.544 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692215712474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 45655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:712.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:712.784 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:35:712.825 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:35:712.876 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.000 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692215712870 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:713.010 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:713.028 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692215712875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:713.063 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:713.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.096 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692215712930 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:35:713.107 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:35:713.221 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.254 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692215713166 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.129, XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:35:713.266 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:35:713.275 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692215713182 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:35:713.303 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:35:713.651 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.664 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692215713645 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:713.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:713.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:713.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692215713668 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:35:713.719 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:35:713.709 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2 To : network-control@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:35:759.923 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:759.986 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692215759806 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:760.014 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:760.331 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:760.359 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692215760304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33215 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:760.369 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:760.627 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:35:760.718 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:35:760.777 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:760.795 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692215760755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:760.803 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:856.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:856.428 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692215856331 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:856.438 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:856.818 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:856.848 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692215856807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 38105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:856.855 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:35:857.136 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:35:857.242 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:35:857.289 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:35:857.337 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692215857268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:35:857.348 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:36:697.047 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:36:697.101 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692216696911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:36:697.113 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:36:697.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:36:697.134 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692216697076 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 39929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:36:697.140 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:36:697.370 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:36:697.422 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:36:697.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:36:697.504 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692216697471 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:36:697.510 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:046.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:046.988 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692217046745 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:047.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:047.068 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:047.080 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692217046965 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 44115 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:047.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:047.187 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:37:047.239 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:37:047.328 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:047.343 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692217047289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:047.349 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:661.425 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:661.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692217661275 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 1584 dequeues = 1584 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 1612 dequeues = 1612 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692188680959 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692217047326 ] max_sm_queue_count = 7 ] 2025-09-01 Mon 02:03:37:661.503 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:37:661.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:661.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692217661397 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 317 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 359 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1210 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 345 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 791, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1584, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:37:661.689 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:37:667.780 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:667.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692217667736 [SYS_NOTICE]: UveVrouterAgent: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 xmpp_peer_list= [ [ [ ip = 10.20.0.193 status = 0 setup_time = 0 primary = 1 ], [ ip = 10.20.0.25 status = 0 setup_time = 0 primary = 0 ], ] ] dns_servers= [ [ (*_iter149) = 10.20.0.193, (*_iter149) = 10.20.0.129, ] ] ] 2025-09-01 Mon 02:03:37:667.850 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:03:37:667.860 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:667.969 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692217667773 [SYS_NOTICE]: VrouterStats: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 in_tpkts = 482 in_bytes = 53375 out_tpkts = 521 out_bytes = 29106 exception_packets = 28 exception_packets_dropped = 11 exception_packets_allowed = 17 raw_vhost_stats= [ name = vhost0 in_pkts = 482 in_bytes = 53375 out_pkts = 521 out_bytes = 29106 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] total_in_bandwidth_utilization = 7.03105e-15 total_out_bandwidth_utilization = 1.68322e-14 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 1297, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 3105, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 447 in_bytes = 25998 out_pkts = 484 out_bytes = 53427 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:37:667.993 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:37:668.001 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:668.039 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692217667794 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.193 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], _iter306->first = 10.20.0.25 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:37:668.052 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:37:715.698 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:715.751 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692217715668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:715.827 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:715.838 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:715.856 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692217715812 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36111 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:715.863 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:37:716.162 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:37:716.210 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:37:716.262 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:37:716.290 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: SANDESH: Send FAILED: 1756692217716255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:37:716.297 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691750893056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:641.691 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:641.759 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692218641569 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:641.773 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:642.285 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:642.305 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692218642255 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 41699 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:642.325 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:642.512 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:38:642.582 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:38:642.644 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:642.676 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692218642635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:38:642.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:38:681.006 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:38:681.257 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Connect : Start Connect timer 10.0.0.38:8086 2025-09-01 Mon 02:03:38:681.354 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:38:681.381 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692218681099 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:38:681.392 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:38:681.401 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:38:681.519 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692218681339 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 317 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 361 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1216 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 347 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 795, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1595, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:38:681.576 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:38:681.607 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:38:681.635 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:38:681.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:38:681.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:38:682.237 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:38:682.272 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Idle 2025-09-01 Mon 02:03:38:682.319 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:682.452 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692218681771 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 318 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 361 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1216 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 347 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 795, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1597, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:38:682.477 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:38:682.491 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:682.508 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692218681812 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:38:682.517 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:38:682.524 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:38:682.550 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:38:682.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692218682309 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 23614 http_port = 8085 start_time = 1756692067658197 collector_name = collector_ip = 10.0.0.131:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.131:8086, (*_iter40) = 10.0.0.249:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 318 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 362 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1216 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 347 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 796, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 1597, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:38:682.670 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:40:141.566 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:141.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220141391 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:141.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:141.666 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:141.677 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220141571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 43527 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:141.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:141.904 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:141.946 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:141.999 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:142.029 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220141991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:142.036 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:142.049 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:142.057 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220141996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:142.062 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:142.086 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:142.110 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220142077 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.193, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:142.123 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:142.133 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:142.147 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220142118 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:40:142.155 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:142.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:142.671 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220142587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:142.682 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:142.692 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:142.722 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220142625 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:142.745 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:142.648 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.193 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:40:191.814 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:191.884 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: SANDESH: Send FAILED: 1756692220191772 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:191.898 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:192.470 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:192.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220192442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 34253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:192.532 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:192.731 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:192.788 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:192.830 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:192.844 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220192822 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:192.851 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:719.590 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:719.657 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220719436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:719.669 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:719.678 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:719.704 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692220719602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect event = Connect Event local_port = 42995 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:719.710 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:719.899 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.129 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:719.943 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:719.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:720.015 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220719987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:720.032 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:720.041 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:720.054 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220719992 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.129 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:720.061 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:720.081 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:720.108 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220720075 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.129 server_addrs= [ [ (*_iter6) = 10.20.0.129:53, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.129, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:720.119 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:720.132 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:720.146 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692220720119 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self Number of connections:4, Expected: 5 ], ] ] ] 2025-09-01 Mon 02:03:40:720.152 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:720.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:720.734 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692220720640 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:720.742 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:720.749 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:720.768 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692220720688 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Idle ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:40:720.778 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:40:720.793 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer an-jenkins-deploy-platform-ansible-os-3995-2/dns:10.20.0.129 Deleted Xmpp Client connection from an-jenkins-deploy-platform-ansible-os-3995-2/dns To : network-dns@contrailsystems.com controller/src/xmpp/xmpp_connection.cc 871 2025-09-01 Mon 02:03:40:771.975 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:772.038 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692220771836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:772.050 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:772.058 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:772.071 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692220771987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 36283 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:772.078 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:40:772.460 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:40:772.522 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:40:772.579 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:40:772.598 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: SANDESH: Send FAILED: 1756692220772570 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:40:772.604 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:128.995 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:129.075 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692221128970 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:129.090 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:129.178 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:129.208 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: SANDESH: Send FAILED: 1756692221129169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect event = Connect Event local_port = 33001 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:129.233 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691734099456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:129.462 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.25 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:41:129.516 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691729901056, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:41:129.560 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:129.578 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692221129553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:129.587 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:424.555 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:424.613 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: SANDESH: Send FAILED: 1756692221424460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:424.624 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691742496256, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:425.072 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:425.112 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692221425047 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect event = Connect Event local_port = 41799 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:425.121 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:425.399 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691766278784, Pid 23614]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.193 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:41:425.518 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691746694656, Pid 23614]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:41:425.569 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:425.593 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692221425553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:425.612 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:425.619 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:425.628 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692221425560 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:41:425.633 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:41:425.639 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:425.661 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: SANDESH: Send FAILED: 1756692221425619 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = Connect ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:41:425.672 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691759289856, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:41:425.708 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:41:425.735 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: SANDESH: Send FAILED: 1756692221425701 [SYS_NOTICE]: NodeStatusUVE: data= [ name = an-jenkins-deploy-platform-ansible-os-3995-2 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:5269, ] ] status = Down description = ], [ type = XMPP name = control-node:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.193 server_addrs= [ [ (*_iter6) = 10.20.0.193:53, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.25 server_addrs= [ [ (*_iter6) = 10.20.0.25:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.193, XMPP:control-node:10.20.0.25, XMPP:dns-server:10.20.0.193, XMPP:dns-server:10.20.0.25, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:41:425.747 UTC an-jenkins-deploy-platform-ansible-os-3995-2 [Thread 139691755091456, Pid 23614]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE