2025-09-01 Mon 01:54:13:205.600 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: vhost interface name : vhost0 2025-09-01 Mon 01:54:13:205.735 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: vhost IP Address : 10.20.0.254/24 2025-09-01 Mon 01:54:13:205.742 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: vhost gateway : 0.0.0.0 2025-09-01 Mon 01:54:13:205.748 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Ethernet ports : ens6 2025-09-01 Mon 01:54:13:205.753 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Loopback IP : 0.0.0.0 2025-09-01 Mon 01:54:13:205.758 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Xmpp Servers : 10.20.0.254:5269 2025-09-01 Mon 01:54:13:205.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Xmpp Authentication : 0 2025-09-01 Mon 01:54:13:205.769 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Cluster-Name : 2025-09-01 Mon 01:54:13:205.775 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: DNS Servers : 10.20.0.254:53 2025-09-01 Mon 01:54:13:205.780 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: DNS client port : 8098 2025-09-01 Mon 01:54:13:205.784 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: DNS timeout : 3000 2025-09-01 Mon 01:54:13:205.789 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: DNS max retries : 2 2025-09-01 Mon 01:54:13:205.793 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Xmpp Dns Authentication : 0 2025-09-01 Mon 01:54:13:205.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TSN Servers : [] 2025-09-01 Mon 01:54:13:205.803 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: COLLECTOR Servers : 10.0.0.33:8086 2025-09-01 Mon 01:54:13:205.808 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Tunnel-Type : MPLSoGRE 2025-09-01 Mon 01:54:13:205.812 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Metadata-Proxy Shared Secret: contrail 2025-09-01 Mon 01:54:13:205.817 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Metadata-Proxy Port : 8097 2025-09-01 Mon 01:54:13:205.821 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Metadata-Proxy SSL Flag : 0 2025-09-01 Mon 01:54:13:205.826 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Linklocal Max System Flows : 16384 2025-09-01 Mon 01:54:13:205.830 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Linklocal Max Vm Flows : 16384 2025-09-01 Mon 01:54:13:205.835 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow cache timeout : 0 2025-09-01 Mon 01:54:13:205.839 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Stale Interface cleanup timeout : 60 2025-09-01 Mon 01:54:13:205.844 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow thread count : 1 2025-09-01 Mon 01:54:13:205.848 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow latency limit : 0 2025-09-01 Mon 01:54:13:205.852 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow index-mgr sm log count : 0 2025-09-01 Mon 01:54:13:205.857 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow add-tokens : 50 2025-09-01 Mon 01:54:13:205.865 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow ksync-tokens : 25 2025-09-01 Mon 01:54:13:205.870 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow del-tokens : 16 2025-09-01 Mon 01:54:13:205.875 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow update-tokens : 16 2025-09-01 Mon 01:54:13:205.879 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Pin flow netlink task to CPU: 2025-09-01 Mon 01:54:13:205.884 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Maximum sessions : 100 2025-09-01 Mon 01:54:13:205.888 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Maximum session aggregates : 8 2025-09-01 Mon 01:54:13:205.893 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Maximum session endpoints : 5 2025-09-01 Mon 01:54:13:205.898 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Fabric SNAT hash table size : 4096 2025-09-01 Mon 01:54:13:205.902 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Flow excluding Router ID in hash :0 2025-09-01 Mon 01:54:13:205.909 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Agent Mode : Vrouter 2025-09-01 Mon 01:54:13:205.914 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Gateway Mode : None 2025-09-01 Mon 01:54:13:205.919 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: DHCP Relay Mode : 0 2025-09-01 Mon 01:54:13:205.923 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Service instance netns cmd : /usr/local/bin/opencontrail-vrouter-netns 2025-09-01 Mon 01:54:13:205.929 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Service instance docker cmd : /usr/local/bin/opencontrail-vrouter-docker 2025-09-01 Mon 01:54:13:205.934 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Service instance workers : 0 2025-09-01 Mon 01:54:13:205.938 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Service instance timeout : 0 2025-09-01 Mon 01:54:13:205.943 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Service instance lb ssl : 2025-09-01 Mon 01:54:13:205.956 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Service instance lbaas auth : 2025-09-01 Mon 01:54:13:205.960 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Bgp as a service port range : 50000-50512 2025-09-01 Mon 01:54:13:205.965 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Services queue limit : 1024 2025-09-01 Mon 01:54:13:205.969 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: BGPAAS max shared sessions for service port : 4 2025-09-01 Mon 01:54:13:205.974 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Sandesh Key file : /etc/contrail/ssl/private/server-privkey.pem 2025-09-01 Mon 01:54:13:205.978 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Sandesh Cert file : /etc/contrail/ssl/certs/server.pem 2025-09-01 Mon 01:54:13:205.983 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Sandesh CA Cert : /etc/contrail/ssl/certs/ca-cert.pem 2025-09-01 Mon 01:54:13:205.987 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Sandesh SSL Enable : 0 2025-09-01 Mon 01:54:13:205.995 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Introspect SSL Enable : 0 2025-09-01 Mon 01:54:13:206.000 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Hypervisor mode : kvm 2025-09-01 Mon 01:54:13:206.090 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Ethernet Port Encap Type : 2025-09-01 Mon 01:54:13:206.097 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: Platform mode : Vrouter on host linux kernel 2025-09-01 Mon 01:54:13:206.390 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: VRouter [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-09-01 Mon 01:54:13:206.425 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Logging: DISABLED -> ENABLED 2025-09-01 Mon 01:54:13:206.469 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-09-01 Mon 01:54:13:219.980 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.020 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> self 2025-09-01 Mon 01:54:13:220.028 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 01:54:13:220.034 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 01:54:13:220.040 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.045 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-access-control-list -> self 2025-09-01 Mon 01:54:13:220.050 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap access-control-list : self -> virtual-network-access-control-list 2025-09-01 Mon 01:54:13:220.055 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap access-control-list : virtual-network-access-control-list -> nil 2025-09-01 Mon 01:54:13:220.061 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.065 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> self 2025-09-01 Mon 01:54:13:220.071 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 01:54:13:220.076 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> virtual-network-network-ipam 2025-09-01 Mon 01:54:13:220.099 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 01:54:13:220.106 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.118 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-qos-config -> self 2025-09-01 Mon 01:54:13:220.125 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-config : self -> virtual-network-qos-config 2025-09-01 Mon 01:54:13:220.129 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-config : virtual-network-qos-config -> nil 2025-09-01 Mon 01:54:13:220.135 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.139 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-security-logging-object -> self 2025-09-01 Mon 01:54:13:220.144 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : self -> virtual-network-security-logging-object 2025-09-01 Mon 01:54:13:220.149 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : virtual-network-security-logging-object -> nil 2025-09-01 Mon 01:54:13:220.155 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.159 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : logical-router-virtual-network -> self 2025-09-01 Mon 01:54:13:220.164 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-router-virtual-network : self -> logical-router-virtual-network 2025-09-01 Mon 01:54:13:220.169 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-router-virtual-network : logical-router-virtual-network -> logical-router-virtual-network 2025-09-01 Mon 01:54:13:220.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-router : self -> logical-router-virtual-network 2025-09-01 Mon 01:54:13:220.179 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-router : logical-router-virtual-network -> nil 2025-09-01 Mon 01:54:13:220.185 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 01:54:13:220.189 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-multicast-policy -> self 2025-09-01 Mon 01:54:13:220.195 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap multicast-policy : self -> virtual-network-multicast-policy 2025-09-01 Mon 01:54:13:220.200 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap multicast-policy : virtual-network-multicast-policy -> nil 2025-09-01 Mon 01:54:13:220.206 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for routing-instance 2025-09-01 Mon 01:54:13:220.219 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> self 2025-09-01 Mon 01:54:13:220.224 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-routing-instance 2025-09-01 Mon 01:54:13:220.229 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-routing-instance 2025-09-01 Mon 01:54:13:220.236 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 01:54:13:220.241 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 01:54:13:220.248 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for security-group 2025-09-01 Mon 01:54:13:220.252 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-group : security-group-access-control-list -> self 2025-09-01 Mon 01:54:13:220.257 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap access-control-list : self -> security-group-access-control-list 2025-09-01 Mon 01:54:13:220.262 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap access-control-list : security-group-access-control-list -> nil 2025-09-01 Mon 01:54:13:220.268 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for tag 2025-09-01 Mon 01:54:13:220.273 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : application-policy-set-tag -> self 2025-09-01 Mon 01:54:13:220.278 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 01:54:13:220.283 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : policy-management-application-policy-set -> application-policy-set-tag 2025-09-01 Mon 01:54:13:220.288 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap policy-management : policy-management-application-policy-set -> nil 2025-09-01 Mon 01:54:13:220.294 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.298 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 01:54:13:220.303 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:220.308 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:220.313 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 01:54:13:220.318 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 01:54:13:220.323 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.328 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 01:54:13:220.332 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 01:54:13:220.340 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine : virtual-machine-interface-virtual-machine -> nil 2025-09-01 Mon 01:54:13:220.345 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.350 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 01:54:13:220.355 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 01:54:13:220.359 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> nil 2025-09-01 Mon 01:54:13:220.364 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 01:54:13:220.374 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:220.378 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:220.383 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 01:54:13:220.388 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 01:54:13:220.393 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.398 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-routing-instance -> self 2025-09-01 Mon 01:54:13:220.403 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface-routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 01:54:13:220.408 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface-routing-instance : virtual-machine-interface-routing-instance -> virtual-machine-interface-routing-instance 2025-09-01 Mon 01:54:13:220.413 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 01:54:13:220.418 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : virtual-machine-interface-routing-instance -> nil 2025-09-01 Mon 01:54:13:220.423 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.427 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-group -> self 2025-09-01 Mon 01:54:13:220.432 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-group : self -> virtual-machine-interface-security-group 2025-09-01 Mon 01:54:13:220.439 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-group : virtual-machine-interface-security-group -> nil 2025-09-01 Mon 01:54:13:220.445 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.449 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : alias-ip-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.455 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap alias-ip : self -> alias-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.460 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap alias-ip : alias-ip-pool-alias-ip -> alias-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.465 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap alias-ip-pool : virtual-network-alias-ip-pool -> alias-ip-pool-alias-ip 2025-09-01 Mon 01:54:13:220.470 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-alias-ip-pool 2025-09-01 Mon 01:54:13:220.484 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-alias-ip-pool -> nil 2025-09-01 Mon 01:54:13:220.491 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.495 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.500 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.505 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.511 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 01:54:13:220.516 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 01:54:13:220.521 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-floating-ip-pool 2025-09-01 Mon 01:54:13:220.525 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 01:54:13:220.530 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 01:54:13:220.536 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.540 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.545 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.552 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.557 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 01:54:13:220.561 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 01:54:13:220.566 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-floating-ip-pool 2025-09-01 Mon 01:54:13:220.571 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 01:54:13:220.575 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 01:54:13:220.581 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.590 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.594 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap floating-ip : instance-ip-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.599 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap instance-ip : instance-ip-virtual-network -> instance-ip-floating-ip 2025-09-01 Mon 01:54:13:220.604 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> instance-ip-virtual-network 2025-09-01 Mon 01:54:13:220.609 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : instance-ip-virtual-network -> nil 2025-09-01 Mon 01:54:13:220.614 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.619 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : instance-ip-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.623 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap instance-ip : self -> instance-ip-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.628 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap instance-ip : instance-ip-virtual-machine-interface -> nil 2025-09-01 Mon 01:54:13:220.633 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.638 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-route-table -> self 2025-09-01 Mon 01:54:13:220.643 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap interface-route-table : self -> virtual-machine-interface-route-table 2025-09-01 Mon 01:54:13:220.650 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap interface-route-table : virtual-machine-interface-route-table -> nil 2025-09-01 Mon 01:54:13:220.655 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.660 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : subnet-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.665 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap subnet : self -> subnet-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.670 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap subnet : subnet-virtual-machine-interface -> nil 2025-09-01 Mon 01:54:13:220.676 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.681 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.686 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> logical-interface-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.692 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 01:54:13:220.697 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 01:54:13:220.701 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 01:54:13:220.707 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.712 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-port-group-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.717 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> virtual-port-group-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.722 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : physical-router-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 01:54:13:220.726 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 01:54:13:220.731 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 01:54:13:220.737 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.741 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.746 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.755 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-as-a-service : bgpaas-bgp-router -> bgpaas-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.761 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-router : self -> bgpaas-bgp-router 2025-09-01 Mon 01:54:13:220.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-router : instance-bgp-router -> bgpaas-bgp-router 2025-09-01 Mon 01:54:13:220.770 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : self -> instance-bgp-router 2025-09-01 Mon 01:54:13:220.775 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : instance-bgp-router -> nil 2025-09-01 Mon 01:54:13:220.780 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.785 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:220.790 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.794 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-as-a-service : bgpaas-health-check -> bgpaas-virtual-machine-interface 2025-09-01 Mon 01:54:13:220.800 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap service-health-check : self -> bgpaas-health-check 2025-09-01 Mon 01:54:13:220.804 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap service-health-check : bgpaas-health-check -> nil 2025-09-01 Mon 01:54:13:220.816 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.820 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-qos-config -> self 2025-09-01 Mon 01:54:13:220.825 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-config : self -> virtual-machine-interface-qos-config 2025-09-01 Mon 01:54:13:220.830 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-config : virtual-machine-interface-qos-config -> nil 2025-09-01 Mon 01:54:13:220.836 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.840 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-logging-object -> self 2025-09-01 Mon 01:54:13:220.845 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : self -> virtual-machine-interface-security-logging-object 2025-09-01 Mon 01:54:13:220.850 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : virtual-machine-interface-security-logging-object -> nil 2025-09-01 Mon 01:54:13:220.856 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.860 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 01:54:13:220.868 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.873 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.879 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.884 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.889 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 01:54:13:220.894 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 01:54:13:220.899 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.903 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 01:54:13:220.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.913 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.917 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 01:54:13:220.922 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bridge-domain : virtual-machine-interface-bridge-domain -> nil 2025-09-01 Mon 01:54:13:220.928 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.933 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-tag -> self 2025-09-01 Mon 01:54:13:220.937 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> virtual-machine-interface-tag 2025-09-01 Mon 01:54:13:220.942 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-interface-tag 2025-09-01 Mon 01:54:13:220.957 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 01:54:13:220.962 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 01:54:13:220.967 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:220.975 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:220.980 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:220.985 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 01:54:13:220.992 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:220.997 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 01:54:13:221.002 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:221.007 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-tag -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:221.012 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> virtual-network-tag 2025-09-01 Mon 01:54:13:221.016 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : application-policy-set-tag -> virtual-network-tag 2025-09-01 Mon 01:54:13:221.021 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.026 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.030 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.035 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.040 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.044 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 01:54:13:221.051 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:221.055 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 01:54:13:221.060 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 01:54:13:221.064 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 01:54:13:221.069 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 01:54:13:221.077 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 01:54:13:221.081 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.086 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.090 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.095 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.099 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.104 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 01:54:13:221.110 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:221.115 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 01:54:13:221.120 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 01:54:13:221.124 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> virtual-machine-interface-sub-interface 2025-09-01 Mon 01:54:13:221.129 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 01:54:13:221.134 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 01:54:13:221.138 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 01:54:13:221.143 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 01:54:13:221.148 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.152 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.157 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.161 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.169 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.173 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 01:54:13:221.179 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:221.184 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : project-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:221.189 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap project : self -> project-virtual-machine-interface 2025-09-01 Mon 01:54:13:221.194 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap project : project-tag -> project-virtual-machine-interface 2025-09-01 Mon 01:54:13:221.199 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> project-tag 2025-09-01 Mon 01:54:13:221.203 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : application-policy-set-tag -> project-tag 2025-09-01 Mon 01:54:13:221.208 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.212 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 01:54:13:221.217 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.222 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.226 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.231 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 01:54:13:221.236 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:221.241 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> self 2025-09-01 Mon 01:54:13:221.246 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.251 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.255 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.260 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> nil 2025-09-01 Mon 01:54:13:221.265 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 01:54:13:221.272 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : logical-router-interface -> self 2025-09-01 Mon 01:54:13:221.277 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-router : self -> logical-router-interface 2025-09-01 Mon 01:54:13:221.282 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-router : logical-router-interface -> nil 2025-09-01 Mon 01:54:13:221.287 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 01:54:13:221.292 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : physical-router-physical-interface -> self 2025-09-01 Mon 01:54:13:221.297 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 01:54:13:221.301 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 01:54:13:221.307 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 01:54:13:221.311 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : virtual-port-group-physical-interface -> self 2025-09-01 Mon 01:54:13:221.317 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-port-group-physical-interface : self -> virtual-port-group-physical-interface 2025-09-01 Mon 01:54:13:221.321 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-port-group-physical-interface : virtual-port-group-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 01:54:13:221.326 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-port-group : self -> virtual-port-group-physical-interface 2025-09-01 Mon 01:54:13:221.331 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> nil 2025-09-01 Mon 01:54:13:221.336 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 01:54:13:221.341 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> self 2025-09-01 Mon 01:54:13:221.345 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : self -> physical-interface-logical-interface 2025-09-01 Mon 01:54:13:221.350 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 01:54:13:221.355 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 01:54:13:221.359 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 01:54:13:221.364 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 01:54:13:221.371 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 01:54:13:221.376 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> logical-interface-virtual-machine-interface 2025-09-01 Mon 01:54:13:221.381 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> nil 2025-09-01 Mon 01:54:13:221.386 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 01:54:13:221.391 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap logical-interface : physical-router-logical-interface -> self 2025-09-01 Mon 01:54:13:221.395 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : self -> physical-router-logical-interface 2025-09-01 Mon 01:54:13:221.400 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : physical-router-logical-interface -> nil 2025-09-01 Mon 01:54:13:221.405 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for physical-router 2025-09-01 Mon 01:54:13:221.409 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-router : physical-router-physical-interface -> self 2025-09-01 Mon 01:54:13:221.414 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : self -> physical-router-physical-interface 2025-09-01 Mon 01:54:13:221.419 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap physical-interface : physical-router-physical-interface -> nil 2025-09-01 Mon 01:54:13:221.425 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 01:54:13:221.429 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 01:54:13:221.434 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 01:54:13:221.439 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> service-port-health-check 2025-09-01 Mon 01:54:13:221.443 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:221.448 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 01:54:13:221.453 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 01:54:13:221.459 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 01:54:13:221.463 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 01:54:13:221.471 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 01:54:13:221.476 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> service-port-health-check 2025-09-01 Mon 01:54:13:221.480 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.485 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.490 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.495 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> port-tuple-interface 2025-09-01 Mon 01:54:13:221.499 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 01:54:13:221.504 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 01:54:13:221.508 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 01:54:13:221.514 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for qos-config 2025-09-01 Mon 01:54:13:221.518 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-config : global-qos-config-qos-config -> self 2025-09-01 Mon 01:54:13:221.524 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap global-qos-config : global-qos-config-qos-config -> nil 2025-09-01 Mon 01:54:13:221.530 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for forwarding-class 2025-09-01 Mon 01:54:13:221.534 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap forwarding-class : forwarding-class-qos-queue -> self 2025-09-01 Mon 01:54:13:221.540 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-queue : self -> forwarding-class-qos-queue 2025-09-01 Mon 01:54:13:221.544 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap qos-queue : forwarding-class-qos-queue -> nil 2025-09-01 Mon 01:54:13:221.550 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 01:54:13:221.554 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : security-logging-object-network-policy -> self 2025-09-01 Mon 01:54:13:221.560 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap network-policy : self -> security-logging-object-network-policy 2025-09-01 Mon 01:54:13:221.564 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap network-policy : security-logging-object-network-policy -> nil 2025-09-01 Mon 01:54:13:221.570 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 01:54:13:221.584 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : security-logging-object-security-group -> self 2025-09-01 Mon 01:54:13:221.589 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-group : self -> security-logging-object-security-group 2025-09-01 Mon 01:54:13:221.594 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-group : security-logging-object-security-group -> nil 2025-09-01 Mon 01:54:13:221.599 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 01:54:13:221.604 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : firewall-policy-security-logging-object -> self 2025-09-01 Mon 01:54:13:221.609 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-security-logging-object : self -> firewall-policy-security-logging-object 2025-09-01 Mon 01:54:13:221.614 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-security-logging-object : firewall-policy-security-logging-object -> firewall-policy-security-logging-object 2025-09-01 Mon 01:54:13:221.619 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : firewall-policy-security-logging-object -> nil 2025-09-01 Mon 01:54:13:221.624 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 01:54:13:221.629 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap security-logging-object : firewall-rule-security-logging-object -> self 2025-09-01 Mon 01:54:13:221.634 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule-security-logging-object : self -> firewall-rule-security-logging-object 2025-09-01 Mon 01:54:13:221.639 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule-security-logging-object : firewall-rule-security-logging-object -> firewall-rule-security-logging-object 2025-09-01 Mon 01:54:13:221.644 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : firewall-rule-security-logging-object -> nil 2025-09-01 Mon 01:54:13:221.651 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for virtual-router 2025-09-01 Mon 01:54:13:221.655 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-router : virtual-router-sub-cluster -> self 2025-09-01 Mon 01:54:13:221.661 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap sub-cluster : self -> virtual-router-sub-cluster 2025-09-01 Mon 01:54:13:221.665 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap sub-cluster : bgp-router-sub-cluster -> virtual-router-sub-cluster 2025-09-01 Mon 01:54:13:221.670 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-router : self -> bgp-router-sub-cluster 2025-09-01 Mon 01:54:13:221.675 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bgp-router : bgp-router-sub-cluster -> nil 2025-09-01 Mon 01:54:13:221.682 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 01:54:13:221.687 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 01:54:13:221.694 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 01:54:13:221.699 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-bridge-domain 2025-09-01 Mon 01:54:13:221.704 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 01:54:13:221.708 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 01:54:13:221.713 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 01:54:13:221.717 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 01:54:13:221.722 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 01:54:13:221.727 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 01:54:13:221.732 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for application-policy-set 2025-09-01 Mon 01:54:13:221.736 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> self 2025-09-01 Mon 01:54:13:221.741 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.746 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.750 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 01:54:13:221.755 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 01:54:13:221.761 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 01:54:13:221.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 01:54:13:221.771 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.775 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.780 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.785 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : firewall-rule-service-group -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.794 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap service-group : self -> firewall-rule-service-group 2025-09-01 Mon 01:54:13:221.799 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap service-group : firewall-rule-service-group -> nil 2025-09-01 Mon 01:54:13:221.804 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 01:54:13:221.809 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 01:54:13:221.813 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.818 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.823 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.827 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : firewall-rule-address-group -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.832 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap address-group : self -> firewall-rule-address-group 2025-09-01 Mon 01:54:13:221.837 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap address-group : address-group-tag -> firewall-rule-address-group 2025-09-01 Mon 01:54:13:221.842 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> address-group-tag 2025-09-01 Mon 01:54:13:221.847 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : address-group-tag -> nil 2025-09-01 Mon 01:54:13:221.852 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 01:54:13:221.856 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 01:54:13:221.861 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.865 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.870 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.875 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : firewall-rule-tag -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.879 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : self -> firewall-rule-tag 2025-09-01 Mon 01:54:13:221.884 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap tag : firewall-rule-tag -> nil 2025-09-01 Mon 01:54:13:221.889 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 01:54:13:221.897 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 01:54:13:221.901 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.906 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.910 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 01:54:13:221.915 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Adding ReactorMap firewall-rule : firewall-policy-firewall-rule -> nil 2025-09-01 Mon 01:54:13:362.256 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.254:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 01:54:13:365.394 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Current receive sock buffer size is 262144 2025-09-01 Mon 01:54:13:365.538 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626727187968, Pid 108698]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU 2025-09-01 Mon 01:54:13:367.185 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Vrouter family is 34 2025-09-01 Mon 01:54:13:368.048 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Flow table size : 20185088 2025-09-01 Mon 01:54:13:368.122 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Flow table size : 161218560 2025-09-01 Mon 01:54:13:368.134 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SetHugePages: Bridge table size:20185088 Flow table size:161218560 2025-09-01 Mon 01:54:13:368.160 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Mem mapping hugepage file:/dev/hugepages/bridge size:20185088 2025-09-01 Mon 01:54:13:368.179 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Mem mapped hugepage file:/dev/hugepages/bridge to addr:0x7fe634800000 2025-09-01 Mon 01:54:13:368.189 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Mem mapping hugepage file:/dev/hugepages/flow size:161218560 2025-09-01 Mon 01:54:13:368.197 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Mem mapped hugepage file:/dev/hugepages/flow to addr:0x7fe62ae00000 2025-09-01 Mon 01:54:13:368.207 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Sending Huge Page configuration to VROUTER 2025-09-01 Mon 01:54:13:425.587 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Huge pages set successfully 2025-09-01 Mon 01:54:13:440.639 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Flow table size : 161218560 2025-09-01 Mon 01:54:13:440.777 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Flow table size : 20185088 2025-09-01 Mon 01:54:13:496.168 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: ROLE : Generator 2025-09-01 Mon 01:54:13:496.203 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: MODULE : contrail-vrouter-agent 2025-09-01 Mon 01:54:13:496.209 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-3994-1 2025-09-01 Mon 01:54:13:496.223 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: NODE TYPE : Compute 2025-09-01 Mon 01:54:13:496.228 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: INSTANCE ID : 0 2025-09-01 Mon 01:54:13:496.233 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: HTTP SERVER PORT : 8085 2025-09-01 Mon 01:54:13:496.240 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-09-01 Mon 01:54:13:496.792 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: HTTP Introspect Init 2025-09-01 Mon 01:54:13:496.989 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8085 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 01:54:13:497.087 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Sandesh Http Server Port: 8085 2025-09-01 Mon 01:54:13:497.149 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: NOT Writing http_port 8085TO : /tmp/contrail-vrouter-agent.107983.http_port 2025-09-01 Mon 01:54:13:497.155 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: CONNECT TO COLLECTOR: 1 2025-09-01 Mon 01:54:13:497.563 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Idle 2025-09-01 Mon 01:54:13:499.101 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: No Client: 1756691653498598 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:13:499.252 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvStart in state Idle 2025-09-01 Mon 01:54:13:499.372 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Disconnect 2025-09-01 Mon 01:54:13:499.441 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvCollectorUpdate in state Disconnect 2025-09-01 Mon 01:54:13:499.712 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:54:13:499.736 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:54:13:499.756 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:54:13:499.762 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:13:499.786 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653499363 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 01:54:13:499.794 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:13:499.801 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:13:499.867 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653499430 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:13:499.883 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:13:499.891 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:13:499.955 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653499471 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:13:499.968 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:13:499.976 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:13:499.989 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653499487 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 01:54:13:499.995 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:13:500.002 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:13:500.042 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653499751 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:13:500.055 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:13:500.063 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:54:13:500.069 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:54:13:500.125 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:13:500.152 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Idle 2025-09-01 Mon 01:54:13:500.204 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:500.218 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653500104 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 01:54:13:500.224 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:13:500.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:54:13:500.272 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:500.341 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653500196 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:13:500.354 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:13:500.514 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:501.983 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691653500434 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 01:54:13:502.093 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 01:54:13:501.984 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface lo flags 65609 2025-09-01 Mon 01:54:13:502.113 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:502.155 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691653500484 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 01:54:13:502.172 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 01:54:13:502.204 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface ens3 flags 69699 2025-09-01 Mon 01:54:13:502.240 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface ens6 flags 69699 2025-09-01 Mon 01:54:13:502.246 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface docker0 flags 4099 2025-09-01 Mon 01:54:13:502.270 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface pkt1 flags 65601 2025-09-01 Mon 01:54:13:502.275 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface pkt3 flags 65601 2025-09-01 Mon 01:54:13:502.280 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface pkt2 flags 65601 2025-09-01 Mon 01:54:13:502.294 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface vhost0 flags 69699 2025-09-01 Mon 01:54:13:502.299 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink interface message RTM_NEWLINK for interface pkt0 flags 69699 2025-09-01 Mon 01:54:13:502.341 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 01:54:13:502.351 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 01:54:13:502.361 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface docker0 2025-09-01 Mon 01:54:13:502.370 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 01:54:13:502.386 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 01:54:13:502.395 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 01:54:13:502.404 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface docker0 2025-09-01 Mon 01:54:13:502.413 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 01:54:13:502.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink address message RTM_NEWADDR for interface pkt0 2025-09-01 Mon 01:54:13:502.470 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.482 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 10.0.0.0/24 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 10.0.0.1/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.502 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 10.0.0.2/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.519 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 10.0.0.4/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.539 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.544 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.553 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Handle netlink route message RTM_NEWROUTE : 172.17.0.0/16 Interface docker0 GW 0.0.0.0 2025-09-01 Mon 01:54:13:502.558 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 01:54:13:502.563 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 01:54:13:502.567 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 01:54:13:502.572 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 01:54:13:502.576 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 01:54:13:502.581 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 01:54:13:502.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 01:54:13:502.590 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 01:54:13:502.594 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 01:54:13:502.609 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.614 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.618 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.623 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.627 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 01:54:13:502.632 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 01:54:13:502.636 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 01:54:13:502.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 01:54:13:502.648 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 01:54:13:502.653 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 01:54:13:502.658 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 01:54:13:502.662 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 01:54:13:502.667 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 01:54:13:502.671 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 01:54:13:522.324 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:9091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 01:54:13:522.461 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Router ID configured. Connection to controller initiated 2025-09-01 Mon 01:54:13:526.059 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:526.106 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691653525994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:526.127 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:526.140 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:526.161 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691653526051 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, Collector connection down, No Configuration for self Number of connections:2, Expected: 3 ], ] ] ] 2025-09-01 Mon 01:54:13:526.167 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:13:526.459 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:526.471 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691653526423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:526.479 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:526.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:526.509 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691653526462 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:13:526.515 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:13:527.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server [fe80::f816:3eff:fe24:f706%vhost0]:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 01:54:13:573.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:573.679 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691653573592 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:573.688 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:573.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:573.779 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:13:573.804 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691653573749 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 40863 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:573.841 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:574.554 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:13:574.607 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:574.622 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691653574598 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:574.640 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:575.616 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:575.643 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691653575605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:575.651 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:575.760 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:13:575.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:13:575.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:575.854 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691653575748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 40365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:575.863 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:13:575.872 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:13:575.881 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691653575831 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:13:575.887 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:14:500.244 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:54:14:500.571 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:54:14:500.590 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:54:14:500.619 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:54:14:500.668 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:14:500.697 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691654500334 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:14:500.712 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:14:500.721 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:54:14:500.727 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:54:14:500.768 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:14:500.792 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Idle 2025-09-01 Mon 01:54:14:500.835 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:500.962 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691654500651 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 8, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:14:500.979 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:14:500.989 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:501.005 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691654500748 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:14:501.015 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:14:501.023 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:54:14:501.051 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:501.132 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691654500824 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 9, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:14:501.144 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:14:546.758 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:546.806 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691654546724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:14:546.830 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:14:546.922 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:14:546.967 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:546.991 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:14:547.015 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691654546905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 38373 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:14:547.031 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:14:547.043 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:547.054 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691654547036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:14:547.059 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:14:547.958 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:547.982 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691654547933 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:14:547.989 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:14:548.114 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:14:548.118 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:548.171 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:14:548.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691654548095 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46333 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:14:548.195 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:14:548.205 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:14:548.230 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691654548199 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:14:548.236 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:366.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:366.484 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691656366385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:16:366.494 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:366.631 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:366.654 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:16:366.677 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691656366602 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 36761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:16:366.716 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:366.755 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:16:366.849 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:366.884 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691656366796 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:16:366.896 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:401.308 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:401.370 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691656401145 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:16:401.388 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:401.546 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:16:401.633 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:401.673 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691656401577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 33035 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:16:401.682 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:401.771 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:16:401.841 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:401.863 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691656401800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:16:401.872 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:16:500.895 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:54:16:502.052 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:54:16:502.072 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:54:16:502.090 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:54:16:502.189 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:16:502.217 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691656501861 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:16:502.226 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:16:502.234 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:54:16:502.247 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:54:16:502.290 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:16:502.315 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 01:54:16:502.357 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:502.460 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691656502172 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 17, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:16:502.485 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:16:502.495 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:502.511 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691656502270 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:16:502.518 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:16:502.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:54:16:502.562 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:16:502.655 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691656502348 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 36, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:16:502.667 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:20:502.450 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:54:20:502.776 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:54:20:502.804 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:54:20:502.838 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:54:20:502.885 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:20:502.924 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691660502548 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:20:502.941 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:20:502.964 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:54:20:502.970 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:54:20:503.009 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:20:503.038 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Idle 2025-09-01 Mon 01:54:20:503.094 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:503.207 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691660502869 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 39, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:20:503.223 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:20:503.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:503.264 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691660502994 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:20:503.274 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:20:503.282 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:54:20:503.315 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:503.404 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691660503084 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 19, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:20:503.416 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:20:649.981 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:650.037 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691660649939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:650.048 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:650.220 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:650.222 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:20:650.265 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691660650190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35719 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:650.300 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:650.329 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:20:650.433 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:650.467 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691660650375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:650.478 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:650.487 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:650.497 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691660650393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:650.502 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:650.509 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:650.548 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691660650463 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:20:650.556 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:20:650.892 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:650.910 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691660650883 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:650.917 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:651.002 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:651.007 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:20:651.013 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691660650997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37909 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:651.034 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:651.065 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:20:651.093 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:651.117 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691660651084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:651.125 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:651.132 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:651.142 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691660651088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:20:651.147 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:20:652.184 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:20:652.218 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691660652153 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:20:652.225 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:21:456.204 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:49852 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:54:28:283.320 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:283.397 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691668283277 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:283.408 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:283.547 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:283.558 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:28:283.593 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668283513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35305 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:283.675 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:283.676 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:28:283.779 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:283.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691668283739 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:283.806 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:283.813 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:283.822 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691668283766 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:283.828 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:503.196 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:54:28:503.531 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:54:28:503.560 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:54:28:503.607 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:54:28:503.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:28:503.670 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668503288 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:28:503.680 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:28:503.689 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:54:28:503.695 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:54:28:503.730 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:28:503.759 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 01:54:28:503.814 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:503.935 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668503616 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 26, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:28:503.982 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:28:503.993 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:504.008 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668503715 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:28:504.016 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:28:504.023 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:54:28:504.057 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:504.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668503805 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 27, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:28:504.192 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:28:642.549 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:642.626 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668642512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:642.637 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:642.755 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:28:642.755 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:642.831 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691668642721 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 38571 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:642.837 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:28:642.839 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:642.936 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:642.957 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668642899 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:642.965 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:28:642.973 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:28:642.982 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691668642919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:28:642.988 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:33:510.552 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:56114 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:54:43:208.019 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:208.888 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691683207970 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 interface_list= [ [ ] ] error_intf_list= [ [ ] ] no_config_intf_list= [ [ (*_iter151) = 00000000-0000-0000-0000-000000000000, ] ] vmi_count= [ active = 1 ] down_interface_count = 1 ] 2025-09-01 Mon 01:54:43:208.921 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 01:54:43:208.938 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:209.502 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691683208280 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [ [ [ name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ] 2025-09-01 Mon 01:54:43:209.539 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: UveVirtualNetworkAgentTrace 2025-09-01 Mon 01:54:43:499.414 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:499.523 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691683499201 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 67 dequeues = 67 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 83 dequeues = 83 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691668503758 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691683208935 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:54:43:499.540 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:43:499.552 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:499.773 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691683499351 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 30, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 67, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:43:499.803 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:43:503.014 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:503.121 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691683502957 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 xmpp_peer_list= [ [ [ ip = 10.20.0.254 status = 0 setup_time = 0 primary = 1 ], ] ] self_ip_list= [ [ (*_iter146) = 10.20.0.254, ] ] build_info = {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} phy_if= [ [ [ name = ens6 mac_address = fa:16:3e:24:f7:06 ], ] ] vhost_if= [ name = vhost0 mac_address = fa:16:3e:24:f7:06 ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_DEBUG log_category = sandesh_http_port = 8085 dns_servers= [ [ (*_iter149) = 10.20.0.254, ] ] vhost_cfg= [ name = vhost0 ip = 10.20.0.254 ip_prefix_len = 24 gateway = 0.0.0.0 ] eth_name= [ [ (*_iter150) = ens6, ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.20.0.254 dns_server_list_cfg= [ [ (*_iter152) = 10.20.0.254:53, ] ] control_node_list_cfg= [ [ (*_iter153) = 10.20.0.254:5269, ] ] hostname_cfg = cn-jenkins-deploy-platform-ansible-os-3994-1 flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 gateway_cfg_list= [ [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [ [ (*_iter155) = 10.0.0.33:8086, ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [ max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-09-01 00:11:15.602301", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-10717-1.novalocal"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name = vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [ [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [ [ (*_iter161) = 0.0.0.0, ] ] ] 2025-09-01 Mon 01:54:43:503.142 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 01:54:43:503.155 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:503.274 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691683503026 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 21 in_bytes = 1014 out_tpkts = 17 out_bytes = 714 exception_packets = 9 exception_packets_dropped = 9 total_flows = 0 aged_flows = 0 uptime = 1756691653206628 phy_if_info= [ [ _iter254->first = ens6 [ speed = -1 duplexity = 255 ], ] ] raw_vhost_stats= [ name = vhost0 in_pkts = 21 in_bytes = 1014 out_pkts = 17 out_bytes = 714 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 0 in_bytes = 0 out_pkts = 21 out_bytes = 1014 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:54:43:503.290 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:54:43:503.304 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:503.328 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691683503061 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:54:43:503.337 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:54:43:691.930 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:691.993 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691683691897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:692.005 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:692.114 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:43:692.191 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:692.193 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:43:692.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691683692156 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:692.256 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:692.268 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:692.278 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691683692259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:692.284 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:692.290 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:692.299 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691683692262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:692.305 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:907.466 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:907.521 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691683907427 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:907.531 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:907.616 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:54:43:907.693 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:43:907.699 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:907.757 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691683907655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 43359 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:907.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:907.774 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:907.790 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691683907718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:907.796 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:43:907.802 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:43:907.811 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691683907722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:54:43:907.817 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:54:44:503.879 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:54:44:504.222 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:54:44:504.245 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:54:44:504.277 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:54:44:504.331 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:54:44:504.357 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691684503984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:44:504.367 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:44:504.376 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:54:44:504.382 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:54:44:504.423 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:54:44:504.452 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 01:54:44:504.498 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:44:504.646 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691684504315 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 34, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:44:504.667 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:44:504.676 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:44:504.693 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691684504403 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:54:44:504.702 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:54:44:504.714 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:54:44:504.753 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:54:44:504.873 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691684504487 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 35, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 81, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:54:44:504.890 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:54:45:501.412 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:32908 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:54:46:582.437 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:36574 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:54:46:588.991 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:36590 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:54:57:560.368 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:60978 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:09:634.916 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:60806 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:12:042.477 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:12:042.555 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691712042433 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:12:042.581 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:12:042.656 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:12:042.661 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:55:12:042.705 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691712042626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 44119 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:12:042.744 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:12:042.759 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:55:12:042.816 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:12:042.828 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691712042791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:12:042.834 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:12:042.841 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:12:042.850 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691712042809 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:12:042.856 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:13:499.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:13:499.673 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691713499440 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 88 dequeues = 88 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691684504452 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691712042840 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:55:13:499.703 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:13:499.712 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:13:499.859 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691713499539 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 42, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:55:13:499.874 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:13:503.757 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:13:503.823 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691713503703 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.11125 five_min_avg = 0.22875 fifteen_min_avg = 0.1775 ] cpu_share = 0.359578 ] ] 2025-09-01 Mon 01:55:13:503.833 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:55:13:503.842 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:13:503.858 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691713503756 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.359578 used_sys_mem = 3296620872 one_min_cpuload = 0.11125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 01:55:13:503.865 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 01:55:13:503.872 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:13:503.994 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691713503783 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 37 in_bytes = 1714 out_tpkts = 32 out_bytes = 1344 exception_packets = 10 exception_packets_dropped = 10 raw_vhost_stats= [ name = vhost0 in_pkts = 37 in_bytes = 1714 out_pkts = 32 out_bytes = 1344 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 0 in_bytes = 0 out_pkts = 37 out_bytes = 1714 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:55:13:504.022 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:55:13:504.035 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:13:504.059 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691713503819 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:55:13:504.066 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:55:14:504.568 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:55:14:504.863 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:55:14:504.894 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:55:14:504.930 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:55:14:504.994 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:55:14:505.022 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691714504656 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:55:14:505.031 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:55:14:505.043 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:55:14:505.050 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:55:14:505.091 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:55:14:505.122 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Idle 2025-09-01 Mon 01:55:14:505.167 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:14:505.321 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691714504969 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 42, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 94, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:55:14:505.347 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:14:505.357 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:14:505.376 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691714505071 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:55:14:505.382 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:55:14:505.390 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:55:14:505.428 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:14:505.549 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691714505158 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 43, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 95, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:55:14:505.563 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:15:497.873 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:15:497.932 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691715497840 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:15:497.950 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:15:498.098 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:55:15:498.107 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:15:498.185 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:55:15:498.194 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691715498069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46499 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:15:498.240 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:15:498.252 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:15:498.263 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691715498232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:15:498.269 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:15:498.275 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:15:498.284 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691715498238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:15:498.290 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:21:958.005 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:40240 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:33:991.046 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:55914 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:41:202.995 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:41:203.060 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691741202956 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:41:203.071 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:41:203.253 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:55:41:203.260 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:41:203.362 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:55:41:203.364 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691741203227 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46545 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:41:203.418 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:41:203.429 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:41:203.440 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691741203424 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:41:203.445 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:41:203.455 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:41:203.463 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691741203429 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:41:203.468 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:42:768.432 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:42:768.486 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691742768394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:42:768.496 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:42:768.593 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:42:768.607 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:55:42:768.639 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691742768565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 43363 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:42:768.679 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:42:768.710 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:55:42:768.805 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:42:768.839 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691742768758 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:42:768.858 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:42:768.866 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:42:768.876 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691742768782 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:55:42:768.881 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:55:43:499.810 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:43:499.883 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691743499650 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 110 dequeues = 110 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 132 dequeues = 132 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691714505121 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691742768865 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:55:43:499.896 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:43:499.905 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:43:500.045 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691743499764 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 51, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 110, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:55:43:500.066 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:43:503.958 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:43:504.113 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691743503926 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 52 in_bytes = 2344 out_tpkts = 47 out_bytes = 1974 raw_vhost_stats= [ name = vhost0 in_pkts = 52 in_bytes = 2344 out_pkts = 47 out_bytes = 1974 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 9.59519e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 0 in_bytes = 0 out_pkts = 52 out_bytes = 2344 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:55:43:504.137 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:55:43:504.147 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:43:504.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691743503989 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:55:43:504.184 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:55:44:505.227 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:55:44:505.503 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:55:44:505.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:55:44:505.615 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:55:44:505.602 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:55:44:505.662 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691744505325 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:55:44:505.671 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:55:44:505.680 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:55:44:505.816 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691744505584 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 51, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 114, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:55:44:505.844 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:44:505.853 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:55:44:505.860 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:55:44:505.901 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:55:44:505.929 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Idle 2025-09-01 Mon 01:55:44:505.989 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:44:506.007 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691744505881 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:55:44:506.013 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:55:44:506.021 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:55:44:506.051 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:55:44:506.176 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691744505980 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 33 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 52, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 116, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:55:44:506.194 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:55:45:968.190 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:56728 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:47:402.609 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:38188 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:47:410.128 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:38196 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:55:58:036.418 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:46654 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:08:473.616 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:08:473.679 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691768473587 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:08:473.690 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:08:473.794 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:08:473.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:56:08:473.841 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691768473767 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 44551 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:08:473.873 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:08:473.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:56:08:473.992 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:08:474.004 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691768473959 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:08:474.012 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:08:474.019 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:08:474.028 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691768473980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:08:474.034 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:09:768.988 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:09:769.081 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691769768954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:09:769.092 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:09:769.169 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:56:09:769.193 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:09:769.235 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691769769153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:09:769.254 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:09:769.242 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:56:09:769.320 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:09:769.353 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691769769305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:09:769.361 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:09:769.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:09:769.378 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691769769311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:09:769.383 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:10:114.724 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:40402 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:13:500.097 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:13:500.189 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691773499891 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 126 dequeues = 126 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 151 dequeues = 151 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691744505928 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691769769368 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:56:13:500.214 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:13:500.224 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:13:500.355 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691773500034 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 61, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 126, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:56:13:500.371 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:13:504.137 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:13:504.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691773504126 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1225 five_min_avg = 0.21125 fifteen_min_avg = 0.175 ] cpu_share = 0.322917 ] ] 2025-09-01 Mon 01:56:13:504.182 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:56:13:504.191 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:13:504.206 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691773504141 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.322917 used_sys_mem = 3296620872 one_min_cpuload = 0.1225 mem_res = 0 ], ] ] ] 2025-09-01 Mon 01:56:13:504.212 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 01:56:13:504.219 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:13:504.320 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691773504169 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 68 in_bytes = 3044 out_tpkts = 62 out_bytes = 2604 exception_packets = 11 exception_packets_dropped = 11 raw_vhost_stats= [ name = vhost0 in_pkts = 68 in_bytes = 3044 out_pkts = 62 out_bytes = 2604 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 0 in_bytes = 0 out_pkts = 68 out_bytes = 3044 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:56:13:504.337 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:56:13:504.346 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:13:504.372 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691773504206 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:56:13:504.379 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:56:14:506.050 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:56:14:506.352 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:56:14:506.368 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:56:14:506.407 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:56:14:506.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:56:14:506.520 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691774506145 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:56:14:506.528 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:56:14:506.537 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:56:14:506.543 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:56:14:506.590 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:56:14:506.627 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 01:56:14:506.674 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:14:506.846 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691774506474 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 61, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 132, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:56:14:506.874 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:14:506.885 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:14:506.901 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691774506564 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:56:14:506.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:56:14:506.915 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:56:14:506.953 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:14:507.069 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691774506663 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 41 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 62, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 133, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:56:14:507.083 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:22:465.781 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:47448 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:34:486.718 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:44550 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:37:364.147 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:37:364.240 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691797364106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:37:364.252 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:37:364.362 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:37:364.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:56:37:364.426 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691797364330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 39415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:37:364.488 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:37:364.517 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:56:37:364.624 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:37:364.660 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691797364571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:37:364.669 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:37:364.678 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:37:364.688 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691797364596 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:37:364.694 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:39:199.511 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:39:199.575 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691799199476 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:39:199.587 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:39:199.702 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:39:199.709 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:56:39:199.749 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691799199676 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 42159 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:39:199.785 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:39:199.816 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:56:39:199.900 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:39:199.913 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691799199864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:39:199.919 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:39:199.925 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:39:199.934 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691799199894 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:56:39:199.948 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:56:43:500.344 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:43:500.421 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691803500183 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 144 dequeues = 144 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 172 dequeues = 172 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691774506625 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691799199924 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:56:43:500.434 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:43:500.451 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:43:500.597 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691803500298 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 68, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 144, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:56:43:500.615 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:43:504.347 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:43:504.524 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691803504321 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 83 in_bytes = 3674 out_tpkts = 78 out_bytes = 3304 raw_vhost_stats= [ name = vhost0 in_pkts = 83 in_bytes = 3674 out_pkts = 78 out_bytes = 3304 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1 in_bytes = 70 out_pkts = 83 out_bytes = 3674 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:56:43:504.548 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:56:43:504.558 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:43:504.584 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691803504371 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:56:43:504.591 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:56:44:506.729 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:56:44:506.997 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:56:44:507.022 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:56:44:507.050 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:56:44:507.098 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:56:44:507.124 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691804506814 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:56:44:507.142 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:56:44:507.151 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:56:44:507.158 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:56:44:507.205 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:56:44:507.236 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 01:56:44:507.303 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:44:507.438 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691804507079 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 68, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 148, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:56:44:507.456 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:44:507.465 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:44:507.480 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691804507178 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:56:44:507.486 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:56:44:507.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:56:44:507.523 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:56:44:507.636 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691804507272 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 69, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 149, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:56:44:507.650 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:56:46:469.907 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:34756 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:48:230.063 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:35258 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:48:236.951 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:35262 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:56:58:527.634 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:54142 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:07:010.055 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:07:010.123 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691827010027 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:07:010.133 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:07:010.256 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:07:010.256 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:57:07:010.302 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691827010228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:07:010.341 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:07:010.356 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:57:07:010.439 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:07:010.473 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691827010396 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:07:010.483 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:07:010.491 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:07:010.501 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691827010414 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:07:010.506 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:08:684.779 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:08:684.844 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691828684748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:08:684.855 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:08:685.012 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:57:08:685.018 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:08:685.093 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:57:08:685.096 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691828684986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 39249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:08:685.123 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:08:685.141 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:08:685.151 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691828685134 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:08:685.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:08:685.186 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:08:685.200 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691828685140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:08:685.208 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:10:570.246 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:52112 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:13:500.601 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:13:500.689 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691833500410 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 160 dequeues = 160 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 191 dequeues = 191 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691804507235 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691828685184 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:57:13:500.704 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:13:500.713 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:13:500.845 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691833500518 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 78, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 160, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:57:13:500.866 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:13:504.537 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:13:504.609 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691833504487 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1025 five_min_avg = 0.195 fifteen_min_avg = 0.17125 ] cpu_share = 0.266667 ] ] 2025-09-01 Mon 01:57:13:504.632 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:57:13:504.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:13:504.657 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691833504516 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.266667 used_sys_mem = 3296620872 one_min_cpuload = 0.1025 mem_res = 0 ], ] ] ] 2025-09-01 Mon 01:57:13:504.663 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 01:57:13:504.670 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:13:504.776 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691833504545 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 98 in_bytes = 4304 out_tpkts = 96 out_bytes = 4060 raw_vhost_stats= [ name = vhost0 in_pkts = 98 in_bytes = 4304 out_pkts = 96 out_bytes = 4060 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 98 out_bytes = 4304 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:57:13:504.792 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:57:13:504.799 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:13:504.823 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691833504572 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:57:13:504.830 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:57:14:507.354 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:57:14:507.616 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:57:14:507.640 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:57:14:507.671 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:57:14:507.711 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:57:14:507.738 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691834507436 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:57:14:507.754 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:57:14:507.763 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:57:14:507.769 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:57:14:507.808 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:57:14:507.834 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Idle 2025-09-01 Mon 01:57:14:507.883 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:14:508.032 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691834507694 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 78, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 166, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:57:14:508.375 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:14:508.395 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:14:508.417 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691834507788 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:57:14:508.424 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:57:14:508.432 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:57:14:508.460 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:14:508.589 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691834507868 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 79, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 167, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:57:14:508.604 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:22:920.546 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:36120 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:35:010.915 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:47830 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:36:225.280 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:36:225.344 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691856225247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:36:225.355 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:36:225.458 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:36:225.467 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:57:36:225.506 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691856225432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35509 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:36:225.546 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:36:225.564 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:57:36:225.650 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:36:225.693 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691856225600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:36:225.721 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:36:225.733 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:36:225.749 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691856225620 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:36:225.757 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:38:060.572 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:38:060.628 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691858060540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:38:060.640 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:38:060.772 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:38:060.774 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:57:38:060.817 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691858060741 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 40761 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:38:060.847 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:38:060.894 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:57:38:061.002 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:38:061.034 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691858060954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:38:061.043 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:38:061.052 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:38:061.062 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691858060981 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:57:38:061.078 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:57:43:500.819 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:43:500.902 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691863500672 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 178 dequeues = 178 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 212 dequeues = 212 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691834507833 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691858061050 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:57:43:500.916 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:43:500.925 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:43:501.129 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691863500784 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 178, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:57:43:501.153 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:43:504.743 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:43:504.913 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691863504714 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 113 in_bytes = 4934 out_tpkts = 111 out_bytes = 4690 raw_vhost_stats= [ name = vhost0 in_pkts = 113 in_bytes = 4934 out_pkts = 111 out_bytes = 4690 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 8.67362e-17 total_out_bandwidth_utilization = 9.1073e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 16, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 168, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 113 out_bytes = 4934 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:57:43:504.932 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:57:43:504.951 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:43:504.976 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691863504747 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:57:43:504.986 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:57:44:507.949 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:57:44:508.219 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:57:44:508.242 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:57:44:508.273 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:57:44:508.323 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:57:44:508.351 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691864508030 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:57:44:508.360 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:57:44:508.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:57:44:508.375 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:57:44:508.415 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:57:44:508.441 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 01:57:44:508.490 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:44:508.622 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691864508301 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 182, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:57:44:508.647 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:44:508.656 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:44:508.671 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691864508394 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:57:44:508.677 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:57:44:508.690 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:57:44:508.717 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:57:44:508.834 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691864508476 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 59 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 183, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:57:44:508.849 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:57:47:070.176 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:36878 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:49:054.850 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:52352 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:49:061.567 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:52364 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:57:59:017.533 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:54458 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:06:735.825 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:06:735.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691886735785 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:06:735.942 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:06:736.039 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:58:06:736.071 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:06:736.122 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:58:06:736.135 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691886736004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 34287 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:06:736.163 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:06:736.176 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:06:736.186 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691886736169 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:06:736.192 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:06:736.228 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:06:736.245 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691886736173 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:06:736.251 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:09:381.182 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:09:381.245 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691889381120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:09:381.257 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:09:381.335 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:09:381.339 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:58:09:381.390 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691889381307 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 40549 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:09:381.418 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:09:381.437 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:58:09:381.503 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:09:381.538 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691889381462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:09:381.547 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:09:381.556 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:09:381.566 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691889381484 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:09:381.572 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:11:112.110 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:52102 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:13:501.118 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:13:501.214 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691893500951 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 194 dequeues = 194 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 231 dequeues = 231 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691864508440 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691889381555 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:58:13:501.243 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:13:501.251 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:13:501.381 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691893501072 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 95, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 194, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:58:13:501.397 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:13:504.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:13:504.986 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691893504882 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.07625 five_min_avg = 0.17125 fifteen_min_avg = 0.165 ] cpu_share = 0.202083 ] ] 2025-09-01 Mon 01:58:13:504.996 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:58:13:505.005 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:13:505.024 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691893504897 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.202083 used_sys_mem = 3296620872 one_min_cpuload = 0.07625 mem_res = 0 ], ] ] ] 2025-09-01 Mon 01:58:13:505.033 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 01:58:13:505.043 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:13:505.206 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691893504928 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 129 in_bytes = 5634 out_tpkts = 126 out_bytes = 5320 exception_packets = 12 exception_packets_dropped = 12 raw_vhost_stats= [ name = vhost0 in_pkts = 129 in_bytes = 5634 out_pkts = 126 out_bytes = 5320 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 129 out_bytes = 5634 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:58:13:505.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:58:13:505.240 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:13:505.265 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691893504976 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:58:13:505.276 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:58:14:508.575 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:58:14:508.874 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:58:14:508.900 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:58:14:508.944 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:58:14:509.004 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:58:14:509.033 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691894508678 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:58:14:509.041 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:58:14:509.050 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:58:14:509.056 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:58:14:509.096 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:58:14:509.125 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Idle 2025-09-01 Mon 01:58:14:509.171 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:14:509.323 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691894508986 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 95, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 200, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:58:14:509.351 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:14:509.365 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:14:509.382 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691894509076 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:58:14:509.389 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:58:14:509.396 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:58:14:509.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:14:509.548 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691894509160 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 96, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 201, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:58:14:509.565 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:23:473.553 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:49774 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:34:276.375 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:34:276.430 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691914276343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:34:276.443 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:34:276.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:34:276.569 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691914276499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35801 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:34:276.531 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:58:34:276.592 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:34:276.659 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:58:34:276.727 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:34:276.761 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691914276685 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:34:276.770 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:34:276.778 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:34:276.789 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691914276705 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:34:276.795 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:35:481.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:58984 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:41:511.627 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:41:511.693 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691921511589 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:41:511.704 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:41:511.828 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:41:511.833 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:58:41:511.870 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691921511800 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 36117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:41:511.916 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:41:511.944 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:58:41:512.077 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:41:512.108 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691921511994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:41:512.114 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:41:512.122 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:41:512.132 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691921512061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:58:41:512.137 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:58:43:501.350 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:43:501.430 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691923501168 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 212 dequeues = 212 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 252 dequeues = 252 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691894509125 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691921512121 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:58:43:501.443 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:43:501.451 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:43:501.589 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691923501299 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 102, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 212, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:58:43:501.614 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:43:505.642 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:43:505.800 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691923505611 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 144 in_bytes = 6264 out_tpkts = 141 out_bytes = 5950 raw_vhost_stats= [ name = vhost0 in_pkts = 144 in_bytes = 6264 out_pkts = 141 out_bytes = 5950 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 0 total_out_bandwidth_utilization = 9.59519e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 144 out_bytes = 6264 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:58:43:505.828 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:58:43:505.841 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:43:505.867 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756691923505657 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:58:43:505.874 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:58:44:509.228 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:58:44:509.534 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:58:44:509.555 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:58:44:509.583 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:58:44:509.647 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:58:44:509.674 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691924509313 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:58:44:509.683 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:58:44:509.701 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:58:44:509.708 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:58:44:509.754 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:58:44:509.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 01:58:44:509.849 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:44:510.001 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691924509617 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 102, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 216, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:58:44:510.024 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:44:510.035 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:44:510.051 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691924509732 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:58:44:510.057 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:58:44:510.064 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:58:44:510.097 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:58:44:510.211 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691924509838 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 72 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 103, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 217, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:58:44:510.366 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:58:47:490.895 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:45590 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:49:873.837 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:38632 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:49:881.304 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:38642 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:58:59:555.242 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:42004 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:59:03:706.884 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:03:706.959 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691943706845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:03:706.971 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:03:707.063 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:03:707.070 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:59:03:707.107 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691943707036 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 33551 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:03:707.136 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:03:707.169 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:59:03:707.266 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:03:707.301 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691943707208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:03:707.320 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:03:707.328 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:03:707.337 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691943707229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:03:707.343 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:10:402.241 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:10:402.432 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:59:10:402.512 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691950402205 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:10:402.572 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:10:402.582 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:10:402.595 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691950402400 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 45845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:10:402.601 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:10:402.612 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:59:10:402.674 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:10:402.711 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691950402653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:10:402.720 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:10:402.736 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:10:402.750 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691950402661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:10:402.768 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:11:552.560 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:54434 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:59:13:501.581 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:13:501.659 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691953501425 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 228 dequeues = 228 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 271 dequeues = 271 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691924509797 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691950402734 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:59:13:501.673 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:13:501.684 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:13:501.826 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691953501542 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 112, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 228, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:59:13:501.848 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:13:505.797 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:13:505.841 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691953505779 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.08625 five_min_avg = 0.16 fifteen_min_avg = 0.16125 ] cpu_share = 0.204167 ] ] 2025-09-01 Mon 01:59:13:505.850 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:59:13:505.859 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:13:505.881 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691953505796 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.204167 used_sys_mem = 3296620872 one_min_cpuload = 0.08625 mem_res = 0 ], ] ] ] 2025-09-01 Mon 01:59:13:505.888 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 01:59:13:505.895 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:13:506.010 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691953505823 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 157 in_bytes = 6810 out_tpkts = 154 out_bytes = 6496 raw_vhost_stats= [ name = vhost0 in_pkts = 157 in_bytes = 6810 out_pkts = 154 out_bytes = 6496 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 157 out_bytes = 6810 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:59:13:506.025 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:59:13:506.033 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:13:506.057 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691953505862 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:59:13:506.064 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:59:14:509.917 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:59:14:510.273 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:59:14:510.298 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:59:14:510.326 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:59:14:510.411 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:59:14:510.454 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691954510025 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:59:14:510.466 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:59:14:510.496 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:59:14:510.505 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:59:14:510.565 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:59:14:510.605 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Idle 2025-09-01 Mon 01:59:14:510.668 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:14:510.808 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691954510388 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 79 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 112, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 234, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:59:14:510.828 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:14:510.837 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:14:510.852 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691954510537 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:59:14:510.858 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:59:14:510.868 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:59:14:510.898 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:14:511.026 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756691954510645 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 113, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 235, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:59:14:511.048 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:23:895.424 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:57540 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:59:32:327.450 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:32:327.526 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756691972327412 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:32:327.538 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:32:327.630 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:59:32:327.631 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:32:327.734 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691972327601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35777 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:32:327.743 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:32:327.785 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:59:32:327.835 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:32:327.878 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691972327811 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:32:327.892 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:32:327.905 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:32:327.921 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756691972327821 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:32:327.941 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:35:958.219 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:51452 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:59:41:722.890 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:41:722.980 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691981722846 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:41:722.993 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:41:723.115 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:41:723.137 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 01:59:41:723.163 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756691981723085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46103 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:41:723.210 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:41:723.248 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:59:41:723.357 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:41:723.386 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691981723308 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:41:723.397 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:41:723.408 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:41:723.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691981723337 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 01:59:41:723.430 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 01:59:43:501.855 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:43:502.000 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691983501677 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 246 dequeues = 246 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 292 dequeues = 292 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691954510604 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756691981723406 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 01:59:43:502.032 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:43:502.046 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:43:502.315 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691983501790 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 119, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 246, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:59:43:502.525 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:43:506.018 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:43:506.167 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691983505994 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 160 in_bytes = 6936 out_tpkts = 157 out_bytes = 6622 raw_vhost_stats= [ name = vhost0 in_pkts = 160 in_bytes = 6936 out_pkts = 157 out_bytes = 6622 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 5 out_bandwidth_usage = 157 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 4.8247e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 89, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 160 out_bytes = 6936 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 01:59:43:506.190 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 01:59:43:506.200 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:43:506.225 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756691983506041 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 01:59:43:506.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 01:59:44:510.751 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 01:59:44:511.077 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 01:59:44:511.098 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 01:59:44:511.126 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 01:59:44:511.202 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 01:59:44:511.231 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691984510855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:59:44:511.239 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:59:44:511.248 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 01:59:44:511.255 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 01:59:44:511.295 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 01:59:44:511.324 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Idle 2025-09-01 Mon 01:59:44:511.374 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:44:511.516 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691984511172 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 119, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 250, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:59:44:511.541 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:44:511.555 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:44:511.576 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691984511275 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 01:59:44:511.587 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 01:59:44:511.594 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 01:59:44:511.623 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 01:59:44:511.738 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756691984511362 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 85 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 120, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 251, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 01:59:44:511.753 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 01:59:47:992.754 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:59936 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:59:50:706.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:41786 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 01:59:50:712.860 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:41798 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:00:144.736 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:52732 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:02:838.019 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:02:838.105 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692002837980 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:02:838.119 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:02:838.230 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:00:02:838.249 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:02:838.325 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692002838200 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37485 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:02:838.332 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:02:838.336 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:02:838.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:02:838.438 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692002838387 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:02:838.445 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:02:838.452 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:02:838.460 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692002838406 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:02:838.466 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:12:277.206 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:40544 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:13:313.495 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:313.557 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692013313457 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:13:313.568 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:13:313.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:313.646 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:00:13:313.685 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692013313612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 39607 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:13:313.734 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:13:313.752 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:13:313.826 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:313.861 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692013313784 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:13:313.870 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:13:313.879 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:313.888 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692013313807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:13:313.894 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:13:502.094 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:502.171 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692013501917 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 262 dequeues = 262 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 311 dequeues = 311 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756691984511323 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692013313877 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:00:13:502.185 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:13:502.196 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:502.333 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692013502033 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 92 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 34 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 129, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 262, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:00:13:502.356 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:13:506.207 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:506.280 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692013506176 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.07 five_min_avg = 0.1425 fifteen_min_avg = 0.155 ] cpu_share = 0.197917 ] ] 2025-09-01 Mon 02:00:13:506.290 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:00:13:506.300 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:506.314 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692013506189 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.197917 used_sys_mem = 3296620872 one_min_cpuload = 0.07 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:00:13:506.321 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:00:13:506.329 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:506.435 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692013506218 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 163 in_bytes = 7062 out_tpkts = 160 out_bytes = 6748 raw_vhost_stats= [ name = vhost0 in_pkts = 163 in_bytes = 7062 out_pkts = 160 out_bytes = 6748 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 163 out_bytes = 7062 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:00:13:506.456 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:00:13:506.467 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:13:506.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692013506272 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:00:13:506.499 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:00:14:511.452 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:00:14:511.790 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:00:14:511.822 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:00:14:511.866 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:00:14:511.920 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:00:14:511.961 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692014511567 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:00:14:511.970 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:00:14:511.979 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:00:14:511.985 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:00:14:512.026 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:14:512.058 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 02:00:14:512.123 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:14:512.296 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692014511903 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 92 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 34 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 129, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 268, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:00:14:512.320 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:14:512.332 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:14:512.348 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692014512006 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:00:14:512.358 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:00:14:512.365 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:00:14:512.393 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:14:512.506 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692014512113 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 93 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 34 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 130, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 269, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:00:14:512.521 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:24:761.510 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:36798 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:30:108.602 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:30:108.665 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692030108563 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:30:108.677 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:30:108.795 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:30:108.798 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:00:30:108.837 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692030108763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 33027 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:30:108.871 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:30:108.907 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:30:109.004 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:30:109.018 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692030108962 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:30:109.024 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:30:109.031 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:30:109.040 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692030108986 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:30:109.046 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:36:873.181 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:42694 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:43:502.354 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:43:502.436 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692043502184 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 276 dequeues = 276 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 328 dequeues = 328 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692014512057 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692030109030 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:00:43:502.450 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:43:502.459 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:43:502.632 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692043502303 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 136 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 134, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 276, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:00:43:502.659 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:43:506.406 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:43:506.532 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692043506390 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 166 in_bytes = 7188 out_tpkts = 163 out_bytes = 6874 raw_vhost_stats= [ name = vhost0 in_pkts = 166 in_bytes = 7188 out_pkts = 163 out_bytes = 6874 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 1.78893e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 196 out_pkts = 166 out_bytes = 7188 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:00:43:506.552 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:00:43:506.560 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:43:506.600 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692043506426 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:00:43:506.610 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:00:44:363.992 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:364.064 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692044363955 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:44:364.076 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:44:364.154 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:364.161 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:00:44:364.197 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692044364123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:44:364.257 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:44:364.301 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:44:364.387 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:364.410 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692044364348 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:44:364.417 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:44:364.425 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:364.438 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692044364374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:44:364.443 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:44:512.188 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:00:44:512.460 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:00:44:512.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:00:44:512.523 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:00:44:512.579 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:00:44:512.605 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692044512277 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:00:44:512.614 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:00:44:512.623 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:00:44:512.629 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:00:44:512.670 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:44:512.718 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 02:00:44:512.778 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:512.912 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692044512565 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 97 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 136, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 284, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:00:44:512.944 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:44:512.955 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:512.971 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692044512648 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:00:44:512.977 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:00:44:512.984 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:00:44:513.013 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:44:513.126 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692044512766 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 98 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 137, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 285, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:00:44:513.140 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:00:48:929.855 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:42294 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:51:518.831 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:36022 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:51:527.207 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:36038 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:00:57:109.324 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:57:109.407 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692057109181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:57:109.429 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:57:109.440 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:00:57:109.444 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:57:109.500 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692057109393 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37401 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:57:109.507 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:57:109.551 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:00:57:109.624 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:57:109.660 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692057109597 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:57:109.669 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:00:57:109.679 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:00:57:109.689 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692057109606 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:00:57:109.695 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:01:116.374 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:35190 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:13:262.066 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:42838 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:13:502.576 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:502.654 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692073502415 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 292 dequeues = 292 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 347 dequeues = 347 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692044512716 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692057109677 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:01:13:502.669 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:13:502.678 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:502.808 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692073502528 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 104 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 144 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 144, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 292, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:13:502.830 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:13:506.590 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:506.663 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692073506561 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.06 five_min_avg = 0.1275 fifteen_min_avg = 0.14875 ] cpu_share = 0.197917 ] ] 2025-09-01 Mon 02:01:13:506.674 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:01:13:506.683 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:506.709 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692073506573 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.197917 used_sys_mem = 3296620872 one_min_cpuload = 0.06 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:01:13:506.716 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:01:13:506.724 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:506.831 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692073506599 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 169 in_bytes = 7314 out_tpkts = 180 out_bytes = 8024 raw_vhost_stats= [ name = vhost0 in_pkts = 169 in_bytes = 7314 out_pkts = 180 out_bytes = 8024 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 18 in_bytes = 1220 out_pkts = 169 out_bytes = 7314 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:01:13:506.845 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:01:13:506.853 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:506.876 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692073506630 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:01:13:506.883 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:01:13:794.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:794.652 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692073794540 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:13:794.663 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:13:794.802 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:794.807 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:13:794.849 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692073794775 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 41323 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:13:794.885 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:13:794.916 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:13:795.024 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:795.038 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692073794987 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:13:795.044 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:13:795.051 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:13:795.060 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692073795014 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:13:795.065 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:14:512.847 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:14:513.180 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:01:14:513.203 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:14:513.236 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:14:513.301 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:14:513.329 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692074512958 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:513.338 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:513.346 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:14:513.353 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:14:513.392 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:14:513.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 02:01:14:513.476 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:513.685 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692074513286 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 105 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 148 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 146, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 302, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:14:513.707 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:14:513.716 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:513.733 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692074513373 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:14:513.739 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:14:513.746 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:14:513.774 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:14:513.890 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692074513466 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 106 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 148 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 147, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 303, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:14:513.905 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:25:694.047 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:42952 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:27:349.783 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:349.972 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:27:350.184 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692087349754 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:350.230 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:350.241 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:350.254 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692087349938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 44295 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:350.260 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:350.286 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:27:350.347 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:350.382 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692087350326 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:350.392 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:27:350.401 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:27:350.411 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692087350333 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:27:350.416 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:37:820.815 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:55420 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:43:502.819 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:502.895 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692103502651 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 310 dequeues = 310 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 368 dequeues = 368 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692074513421 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692087350399 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:01:43:502.926 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:43:502.935 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:503.065 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692103502769 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 109 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 152 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 151, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 310, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:43:503.211 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:43:506.760 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:506.912 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692103506738 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 172 in_bytes = 7440 out_tpkts = 187 out_bytes = 8374 raw_vhost_stats= [ name = vhost0 in_pkts = 172 in_bytes = 7440 out_pkts = 187 out_bytes = 8374 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 8.99888e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 166, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 22 in_bytes = 1444 out_pkts = 172 out_bytes = 7440 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:01:43:506.943 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:01:43:506.953 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:43:506.978 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692103506777 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:01:43:506.985 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:01:44:513.533 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:01:44:513.817 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:01:44:513.852 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:01:44:513.893 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:01:44:513.979 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:01:44:514.018 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692104513623 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:44:514.027 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:44:514.037 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:01:44:514.043 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:01:44:514.087 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:44:514.117 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Idle 2025-09-01 Mon 02:01:44:514.185 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:514.363 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692104513952 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 109 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 152 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 151, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 314, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:44:514.385 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:44:514.397 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:514.413 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692104514065 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:01:44:514.422 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:01:44:514.430 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:01:44:514.459 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:44:514.573 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692104514155 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 110 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 152 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 152, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 315, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:01:44:514.590 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:01:45:385.195 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:385.259 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692105385154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:385.270 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:45:385.382 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:385.391 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:45:385.426 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692105385354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 44343 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:385.480 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:45:385.491 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:45:385.582 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:385.596 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692105385543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:385.602 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:45:385.610 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:45:385.619 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692105385562 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:45:385.624 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:49:929.764 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:60292 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:52:349.823 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:58390 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:52:357.204 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:58398 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:01:59:480.508 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:480.574 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692119480472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:480.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:59:480.684 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:480.685 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:01:59:480.747 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692119480654 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 43907 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:480.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:59:480.772 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:01:59:480.863 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:480.898 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692119480823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:480.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:01:59:480.929 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:01:59:480.940 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692119480842 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:01:59:480.946 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:02:090.048 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:50666 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:13:195.754 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:195.831 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692133195712 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:195.845 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:195.950 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:13:195.954 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:196.012 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692133195909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:196.022 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:196.023 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:13:196.197 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:196.221 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692133196107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:196.227 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:196.234 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:196.243 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692133196126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:13:196.249 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:13:503.096 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:503.173 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692133502880 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 330 dequeues = 330 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 391 dequeues = 391 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692104514116 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692133196233 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:02:13:503.186 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:13:503.195 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:503.336 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692133503006 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 117 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 164 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 162, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 330, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:13:503.360 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:13:506.936 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:507.006 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692133506896 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.055 five_min_avg = 0.11375 fifteen_min_avg = 0.1425 ] cpu_share = 0.2 ] ] 2025-09-01 Mon 02:02:13:507.027 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:13:507.037 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:507.052 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692133506905 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.2 used_sys_mem = 3296620872 one_min_cpuload = 0.055 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:02:13:507.059 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:02:13:507.070 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:507.177 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692133506941 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 176 in_bytes = 7636 out_tpkts = 207 out_bytes = 9678 raw_vhost_stats= [ name = vhost0 in_pkts = 176 in_bytes = 7636 out_pkts = 207 out_bytes = 9678 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 39 in_bytes = 2622 out_pkts = 176 out_bytes = 7636 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:02:13:507.192 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:13:507.202 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:13:507.225 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692133506967 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:02:13:507.232 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:02:14:110.225 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:53070 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:14:514.248 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:14:514.600 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:02:14:514.623 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:14:514.658 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:14:514.758 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:14:514.823 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692134514369 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:14:514.850 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:14:514.864 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:14:514.873 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:14:514.946 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:14:514.996 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Idle 2025-09-01 Mon 02:02:14:515.081 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:14:515.334 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692134514737 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 118 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 164 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 163, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 336, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:14:515.359 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:14:515.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:14:515.386 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692134514903 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:14:515.392 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:14:515.411 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:14:515.451 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:14:515.663 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692134515063 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 119 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 164 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 164, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 337, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:14:515.691 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:26:584.629 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:45600 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:31:611.044 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:611.120 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692151611009 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:611.132 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:611.292 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:31:611.298 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:611.371 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:31:611.374 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692151611256 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 41949 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:611.396 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:611.426 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:611.464 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692151611407 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:611.489 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:31:611.497 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:31:611.507 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692151611414 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:31:611.512 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:38:696.780 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:46142 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:40:466.312 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:466.388 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692160466272 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:466.400 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:466.491 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:466.509 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:40:466.532 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692160466463 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 33227 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:466.571 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:466.603 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:466.684 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:466.696 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692160466649 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:466.702 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:466.709 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:466.731 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692160466672 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:466.737 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:43:503.277 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:503.345 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692163503115 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 348 dequeues = 348 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 412 dequeues = 412 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692134514995 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692160466708 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:02:43:503.359 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:43:503.368 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:503.497 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692163503231 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 123 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 172 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 348, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:43:503.519 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:43:507.122 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:507.260 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692163507100 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 179 in_bytes = 7762 out_tpkts = 224 out_bytes = 10940 exception_packets = 13 exception_packets_dropped = 13 raw_vhost_stats= [ name = vhost0 in_pkts = 179 in_bytes = 7762 out_pkts = 224 out_bytes = 10940 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 1.66967e-15 total_out_bandwidth_utilization = 2.27682e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 308, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 42, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 53 in_bytes = 3758 out_pkts = 179 out_bytes = 7762 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:02:43:507.279 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:43:507.288 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:43:507.313 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692163507145 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:02:43:507.322 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:02:44:515.102 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:44:515.359 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:02:44:515.403 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:44:515.434 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:44:515.484 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:44:515.514 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692164515187 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:44:515.522 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:44:515.532 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:44:515.538 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:44:515.580 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:44:515.607 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 02:02:44:515.665 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:515.803 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692164515458 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 78 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 123 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 172 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 170, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 352, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:44:515.836 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:44:515.849 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:515.874 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692164515559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:44:515.883 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:44:515.892 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:44:515.932 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:44:516.065 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692164515645 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 78 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 124 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 172 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 353, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:44:516.102 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:50:719.346 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:50834 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:53:178.503 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:33052 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:53:185.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:33054 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:59:421.599 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:421.672 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692179421561 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:421.684 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:421.760 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:59:421.767 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:421.811 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692179421736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 35481 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:421.820 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:421.824 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:59:421.877 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:421.888 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692179421868 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:421.893 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:59:421.900 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:59:421.919 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692179421882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:59:421.925 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:02:749.258 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:60702 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:09:896.886 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:09:896.975 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692189896852 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:09:897.001 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:09:897.073 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:09:897.076 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:09:897.130 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:09:897.138 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692189897045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 46743 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:09:897.161 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:09:897.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:09:897.186 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692189897165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:09:897.192 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:09:897.199 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:09:897.207 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692189897175 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:09:897.213 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:13:503.524 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:527.283 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692193503356 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 364 dequeues = 364 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 431 dequeues = 431 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692164515606 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692189897198 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:03:13:527.335 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:13:527.349 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:527.491 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692193503470 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 130 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 180 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 179, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 364, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:13:527.508 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:13:527.522 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:527.549 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692193507273 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.0625 five_min_avg = 0.10375 fifteen_min_avg = 0.1375 ] cpu_share = 0.197917 ] ] 2025-09-01 Mon 02:03:13:527.557 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:13:527.564 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:527.578 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692193507287 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.197917 used_sys_mem = 3296620872 one_min_cpuload = 0.0625 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:03:13:527.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:03:13:527.592 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:527.694 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692193507316 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 182 in_bytes = 7888 out_tpkts = 230 out_bytes = 11276 raw_vhost_stats= [ name = vhost0 in_pkts = 182 in_bytes = 7888 out_pkts = 230 out_bytes = 11276 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 56 in_bytes = 3968 out_pkts = 182 out_bytes = 7888 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:13:527.707 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:13:527.714 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:13:527.738 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692193507357 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:13:527.748 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:14:515.713 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:14:515.980 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:03:14:516.008 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:14:516.039 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:14:516.122 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:14:516.151 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692194515794 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:14:516.161 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:14:516.169 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:14:516.175 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:14:516.215 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:14:516.245 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 02:03:14:516.296 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:516.429 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692194516066 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 130 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 180 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 179, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 370, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:14:516.454 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:14:516.463 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:516.478 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692194516195 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:14:516.485 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:14:516.492 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:14:516.520 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:14:516.632 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692194516281 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 131 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 180 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 180, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 371, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:14:516.646 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:14:871.160 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:53538 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:27:384.818 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:54668 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:28:582.087 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:582.158 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692208582055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:582.170 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:582.247 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:582.250 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:28:582.292 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692208582218 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 44019 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:582.321 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:582.341 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:28:582.391 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:582.434 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692208582367 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:582.447 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:28:582.459 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:28:582.474 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692208582374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:28:582.482 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:327.375 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:327.437 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692219327343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:327.448 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:327.584 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:39:327.587 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:327.665 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692219327553 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 43335 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:327.688 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:327.672 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:39:327.742 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:327.753 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692219327734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:327.759 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:327.765 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:39:327.774 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692219327744 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:39:327.779 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:39:529.928 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:58914 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:43:503.774 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:503.853 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692223503617 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 382 dequeues = 382 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 452 dequeues = 452 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692194516244 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692219327764 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:03:43:503.866 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:43:503.874 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:504.019 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692223503733 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 136 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 188 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 187, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 382, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:43:504.047 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:43:507.505 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:507.670 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692223507482 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 185 in_bytes = 8014 out_tpkts = 236 out_bytes = 11612 raw_vhost_stats= [ name = vhost0 in_pkts = 185 in_bytes = 8014 out_pkts = 236 out_bytes = 11612 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 3.03577e-16 total_out_bandwidth_utilization = 1.78893e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 56, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 59 in_bytes = 4178 out_pkts = 185 out_bytes = 8014 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:43:507.693 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:43:507.703 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:43:507.728 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692223507517 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:43:507.735 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:44:516.364 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:44:516.626 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:03:44:516.652 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:44:516.681 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:44:516.741 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:44:516.767 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692224516454 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:44:516.786 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:44:516.795 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:44:516.801 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:44:516.845 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:44:516.889 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 02:03:44:516.949 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:517.097 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692224516718 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 86 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 136 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 188 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 187, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 386, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:44:517.120 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:44:517.132 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:517.148 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692224516825 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:44:517.155 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:44:517.162 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:44:517.187 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:44:517.300 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692224516939 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 86 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 188 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 188, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 387, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:44:517.318 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:51:623.246 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:48098 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:53:998.124 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:36158 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:54:005.616 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:36174 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:00:442.562 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:442.618 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692240442528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:442.629 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:442.752 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:00:442.761 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:442.831 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692240442725 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 41439 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:442.848 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:442.804 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:00:442.944 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:442.970 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692240442934 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:442.988 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:00:442.996 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:00:443.006 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692240442941 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:00:443.012 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:03:763.306 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:39106 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:11:187.893 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:187.979 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692251187858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:187.991 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:188.117 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:11:188.129 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:188.212 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692251188068 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 34161 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:188.220 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:11:188.230 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:188.317 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:188.330 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692251188283 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:188.336 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:188.343 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:11:188.365 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692251188313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:11:188.373 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:13:504.024 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:504.112 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692253503844 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 398 dequeues = 398 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 471 dequeues = 471 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692224516888 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692251188342 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:04:13:504.127 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:13:504.138 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:504.268 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692253503955 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 88 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 143 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 196, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 398, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:13:504.289 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:13:507.697 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:507.769 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692253507676 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.055 five_min_avg = 0.0925 fifteen_min_avg = 0.13125 ] cpu_share = 0.195833 ] ] 2025-09-01 Mon 02:04:13:507.784 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:13:508.066 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:508.132 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692253507691 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.195833 used_sys_mem = 3296620872 one_min_cpuload = 0.055 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:04:13:508.144 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:04:13:508.153 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:508.259 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692253507724 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 188 in_bytes = 8140 out_tpkts = 251 out_bytes = 12734 raw_vhost_stats= [ name = vhost0 in_pkts = 188 in_bytes = 8140 out_pkts = 251 out_bytes = 12734 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 71 in_bytes = 5174 out_pkts = 188 out_bytes = 8140 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:13:508.275 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:13:508.283 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:13:508.308 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692253507775 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:13:508.316 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:14:517.010 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:14:517.257 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:04:14:517.280 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:14:517.309 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:14:517.383 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:14:517.410 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692254517087 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:14:517.420 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:517.438 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:14:517.444 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:14:517.488 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:14:517.517 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 02:04:14:517.563 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:517.696 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692254517362 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 143 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 196, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 404, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:14:517.715 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:14:517.726 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:517.742 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692254517468 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:14:517.748 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:14:517.754 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:14:517.778 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:14:517.890 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692254517553 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 144 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 197, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 405, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:14:517.916 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:15:856.783 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:51288 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:28:302.405 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:49324 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:32:033.176 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:033.250 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692272033142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:033.263 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:033.423 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:033.423 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:32:033.479 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692272033388 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 33415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:033.510 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:033.525 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:32:033.620 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:033.661 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692272033558 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:033.673 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:32:033.684 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:32:033.698 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692272033590 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:32:033.719 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:458.556 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:458.614 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692278458520 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:458.626 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:458.751 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:458.766 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:38:458.803 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692278458723 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37387 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:458.849 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:458.867 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:38:458.935 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:458.947 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692278458908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:458.954 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:38:458.961 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:38:458.970 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692278458926 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:38:458.976 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:40:437.961 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:43430 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:43:504.282 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:504.357 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692283504119 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 416 dequeues = 416 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 492 dequeues = 492 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692254517517 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692278458960 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:04:43:504.371 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:43:504.379 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:504.507 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692283504231 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 149 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 204 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 204, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 416, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:43:504.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:43:507.909 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:508.061 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692283507873 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 191 in_bytes = 8266 out_tpkts = 256 out_bytes = 13000 raw_vhost_stats= [ name = vhost0 in_pkts = 191 in_bytes = 8266 out_pkts = 256 out_bytes = 13000 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 136 out_bandwidth_usage = 35 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 8.18573e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 151, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 73 in_bytes = 5314 out_pkts = 191 out_bytes = 8266 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:43:508.101 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:43:508.111 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:43:508.136 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692283507915 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:43:508.147 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:44:517.601 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:44:517.863 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:04:44:517.887 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:44:517.927 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:44:517.979 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:44:518.007 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692284517686 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:44:518.016 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:44:518.025 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:44:518.032 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:44:518.072 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:44:518.101 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Idle 2025-09-01 Mon 02:04:44:518.166 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:518.299 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692284517957 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 149 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 204 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 204, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 420, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:44:518.323 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:44:518.335 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:518.351 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692284518052 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:44:518.359 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:44:518.368 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:44:518.392 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:44:518.506 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692284518135 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 150 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 204 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 205, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 421, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:44:518.521 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:52:560.319 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:59918 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:54:830.287 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:57328 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:54:837.217 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:57332 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:00:113.772 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:113.834 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692300113738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:113.845 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:113.967 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:113.972 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:00:114.012 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692300113937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 41315 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:114.052 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:114.083 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:00:114.169 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:114.202 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692300114123 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:114.212 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:00:114.221 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:00:114.233 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692300114144 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:00:114.239 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:04:609.145 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:33050 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:10:049.100 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:049.177 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692310049065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:049.190 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:049.299 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:049.308 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:10:049.348 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692310049269 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 37655 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:049.403 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:049.417 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:10:049.493 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:049.532 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692310049450 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:049.541 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:10:049.551 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:10:049.560 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692310049470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:10:049.565 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:13:504.601 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:504.707 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692313504395 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 432 dequeues = 432 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 511 dequeues = 511 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692284518100 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692310049549 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:05:13:504.724 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:13:504.735 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:504.900 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692313504544 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 156 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 212 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 213, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 432, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:13:504.917 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:13:508.069 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:508.145 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692313508040 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.065 five_min_avg = 0.09125 fifteen_min_avg = 0.1275 ] cpu_share = 0.202083 ] ] 2025-09-01 Mon 02:05:13:508.156 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:13:508.165 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:508.184 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692313508051 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.202083 used_sys_mem = 3296620872 one_min_cpuload = 0.065 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:05:13:508.194 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:05:13:508.204 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:508.313 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692313508106 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 194 in_bytes = 8392 out_tpkts = 260 out_bytes = 13196 raw_vhost_stats= [ name = vhost0 in_pkts = 194 in_bytes = 8392 out_pkts = 260 out_bytes = 13196 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 74 in_bytes = 5384 out_pkts = 194 out_bytes = 8392 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:13:508.332 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:13:508.343 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:13:508.367 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692313508151 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:13:508.373 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:14:518.208 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:14:518.521 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:05:14:518.546 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:14:518.576 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:14:518.642 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:14:518.668 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692314518321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:14:518.677 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:518.686 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:14:518.692 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:14:518.731 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:14:518.761 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Idle 2025-09-01 Mon 02:05:14:518.806 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:518.951 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692314518628 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 98 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 156 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 212 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 213, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 438, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:14:518.972 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:14:518.982 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:518.998 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692314518712 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:14:519.007 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:14:519.024 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:14:519.054 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:14:519.168 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692314518796 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 98 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 212 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 214, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 439, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:14:519.183 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:16:715.021 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:38306 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:27:654.317 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:654.369 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692327654286 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:654.380 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:654.468 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:654.473 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:27:654.481 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692327654444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 45689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:654.516 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:654.544 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:27:654.606 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:654.636 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692327654591 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:654.645 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:27:654.657 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:27:654.667 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692327654599 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:27:654.672 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:29:134.979 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:59172 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:38:939.683 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:939.757 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692338939646 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:939.768 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:939.868 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:939.895 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:38:939.926 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692338939841 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 42845 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:939.963 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:939.995 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:38:940.137 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:940.182 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692338940070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:940.194 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:38:940.206 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:38:940.221 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692338940124 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:38:940.230 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:109.536 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:58400 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:43:504.838 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:504.919 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692343504677 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 450 dequeues = 450 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 532 dequeues = 532 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692314518760 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692338940204 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:05:43:504.933 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:43:504.944 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:505.093 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692343504781 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 162 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 221, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 450, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:43:505.117 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:43:508.265 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:508.427 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692343508241 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 197 in_bytes = 8518 out_tpkts = 264 out_bytes = 13392 raw_vhost_stats= [ name = vhost0 in_pkts = 197 in_bytes = 8518 out_pkts = 264 out_bytes = 13392 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 9.75782e-17 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 18, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 75 in_bytes = 5454 out_pkts = 197 out_bytes = 8518 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:43:508.448 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:43:508.458 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:43:508.483 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692343508295 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:43:508.489 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:44:518.905 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:44:519.206 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:05:44:519.220 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:44:519.255 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:44:519.337 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:44:519.364 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692344519001 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:519.373 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:519.382 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:44:519.397 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:44:519.443 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:44:519.476 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Idle 2025-09-01 Mon 02:05:44:519.538 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:519.675 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692344519316 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 102 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 162 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 221, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 454, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:44:519.696 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:44:519.707 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:519.723 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692344519422 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:44:519.729 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:44:519.737 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:44:519.764 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:44:519.878 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692344519521 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 102 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 163 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 222, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 455, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:44:519.908 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:53:188.103 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:33666 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:55:658.344 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:37776 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:55:665.826 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:37792 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:57:894.813 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:894.895 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692357894771 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:894.906 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:895.044 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:895.046 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:57:895.095 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692357895012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 39383 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:895.140 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:895.151 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:57:895.215 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:895.247 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692357895183 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:895.258 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:57:895.269 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:57:895.280 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692357895201 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:57:895.304 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:05:356.996 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:60878 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:11:070.331 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:070.413 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692371070292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:070.424 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:070.520 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:070.533 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:11:070.569 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692371070493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 44253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:070.612 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:070.642 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:11:070.730 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:070.742 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692371070692 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:070.748 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:11:070.754 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:11:070.763 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692371070714 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:11:070.769 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:13:505.118 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:505.201 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692373504953 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 466 dequeues = 466 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 551 dequeues = 551 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692344519475 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692371070753 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:06:13:505.215 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:13:505.224 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:505.355 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692373505075 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 169 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 230, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 466, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:13:505.378 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:13:508.444 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:508.486 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692373508425 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 998346424 node_type = ] meminfo= [ virt = 32742 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.07625 five_min_avg = 0.09125 fifteen_min_avg = 0.125 ] cpu_share = 0.195801 ] ] 2025-09-01 Mon 02:06:13:508.495 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:13:508.505 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:508.521 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692373508443 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 cpu_info= [ [ [ mem_virt = 32742 cpu_share = 0.195801 used_sys_mem = 3296620872 one_min_cpuload = 0.07625 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:06:13:508.533 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:06:13:508.541 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:508.641 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692373508471 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 200 in_bytes = 8644 out_tpkts = 268 out_bytes = 13588 raw_vhost_stats= [ name = vhost0 in_pkts = 200 in_bytes = 8644 out_pkts = 268 out_bytes = 13588 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 76 in_bytes = 5524 out_pkts = 200 out_bytes = 8644 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:13:508.654 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:13:508.663 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:13:508.688 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692373508512 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:13:508.694 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:14:519.585 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:14:519.836 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:06:14:519.861 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:14:519.899 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:14:519.968 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:14:519.994 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692374519672 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:14:520.001 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:14:520.011 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:14:520.017 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:14:520.064 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:14:520.134 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Idle 2025-09-01 Mon 02:06:14:520.191 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:520.324 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692374519930 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 106 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 169 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 230, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 472, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:14:520.342 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:14:520.350 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:520.366 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692374520039 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:14:520.372 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:14:520.379 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:14:520.404 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:14:520.515 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692374520172 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 106 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 170 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 231, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 473, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:14:520.529 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:17:443.970 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:38444 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:25:975.386 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:975.451 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692385975352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:975.463 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:975.566 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:975.577 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:25:975.615 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692385975538 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 34249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:975.643 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:975.686 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:975.781 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:975.818 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692385975730 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:975.828 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:25:975.846 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:975.856 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: SANDESH: Send FAILED: 1756692385975751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:25:975.862 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626739783168, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:29:900.372 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:54148 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:31:965.100 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:35278 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:34:071.946 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:35294 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:39:150.868 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:150.951 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: SANDESH: Send FAILED: 1756692399150834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:150.963 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626743981568, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:151.074 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:39:151.077 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:151.156 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692399151042 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 36643 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:151.170 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:151.159 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:39:151.268 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:151.280 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692399151229 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:151.287 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:39:151.293 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:39:151.302 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: SANDESH: Send FAILED: 1756692399151250 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:39:151.307 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:43:505.372 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:505.441 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692403505201 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 484 dequeues = 484 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 572 dequeues = 572 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692374520133 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692399151292 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:06:43:505.464 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:43:505.479 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:505.609 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692403505319 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 176 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 239, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 484, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:43:505.624 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:43:508.654 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:508.801 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692403508623 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 in_tpkts = 203 in_bytes = 8770 out_tpkts = 273 out_bytes = 13854 raw_vhost_stats= [ name = vhost0 in_pkts = 203 in_bytes = 8770 out_pkts = 273 out_bytes = 13854 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 1.51788e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 28, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 78 in_bytes = 5664 out_pkts = 203 out_bytes = 8770 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:43:508.823 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:43:508.835 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:43:508.861 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: SANDESH: Send FAILED: 1756692403508675 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.254 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:43:508.867 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626731386368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:44:520.244 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:44:520.510 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Connect : Start Connect timer 10.0.0.33:8086 2025-09-01 Mon 02:06:44:520.533 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:44:520.562 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:44:520.647 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:44:520.675 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692404520338 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:44:520.683 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:44:520.692 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:44:520.698 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:44:520.736 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:44:520.762 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Idle 2025-09-01 Mon 02:06:44:520.808 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:520.950 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692404520623 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 176 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 239, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 488, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:44:520.972 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:44:520.982 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:520.997 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692404520718 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.254 server_addrs= [ [ (*_iter6) = 10.20.0.254:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.254, XMPP:dns-server:10.20.0.254, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:44:521.006 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:44:521.013 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:44:521.039 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:44:521.151 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: SANDESH: Send FAILED: 1756692404520798 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3994-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 108698 http_port = 8085 start_time = 1756691653497573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 177 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 240, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 489, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:44:521.165 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626748179968, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:52:975.920 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:975.980 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: SANDESH: Send FAILED: 1756692412975875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:975.992 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626735584768, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:976.135 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:976.136 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.254 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:52:976.180 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692412976100 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect event = Connect Event local_port = 38793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:976.230 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:976.262 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626756576768, Pid 108698]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:52:976.348 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:976.362 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692412976315 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:976.368 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:976.375 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:976.384 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: SANDESH: Send FAILED: 1756692412976335 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.254 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:976.389 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626752378368, Pid 108698]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:56:474.275 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:37204 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:56:481.136 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.254:8085::10.20.0.254:37206 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:00:289.857 UTC cn-jenkins-deploy-platform-ansible-os-3994-1 [Thread 140626763565696, Pid 108698]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:8085::10.0.0.33:34450 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255